[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:52 dist-test-slave-m1rv krb5kdc[9032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:52 dist-test-slave-m1rv krb5kdc[9032](info): set up 2 sockets
Sep 29 02:57:52 dist-test-slave-m1rv krb5kdc[9032](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 02:57:53.729710 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.003s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:57:53 dist-test-slave-m1rv krb5kdc[9032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:57:53.908944 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:36009
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:36009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:54.229560 9045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:54.229862 9045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:54.229954 9045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:54.237275 9045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:54.237392 9045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:54.237439 9045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:54.237490 9045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:54.237535 9045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:54.247634 9045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:36009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:36009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:54.248941 9045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:54.250305 9045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:54.262790 9053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:54.262813 9050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:54.263180 9051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:54.263236 9045 server_base.cc:1034] running on GCE node
I20260929 02:57:54.264312 9045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:54.265802 9045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:54.267027 9045 hybrid_clock.cc:648] HybridClock initialized: now 1790650674266982 us; error 32 us; skew 500 ppm
I20260929 02:57:54.267372 9045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:54 dist-test-slave-m1rv krb5kdc[9032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:54.270498 9045 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:57:54.271938 9045 webserver.cc:460] Webserver started at http://127.8.208.126:39649/ using document root <none> and password file <none>
I20260929 02:57:54.272629 9045 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:54.272773 9045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:54.273070 9045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:54.276077 9045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "3e262df1a9c749629300197f107996a5"
format_stamp: "Formatted at 2026-09-29 02:57:54 on dist-test-slave-m1rv"
server_key: "d3d33028917eeb06923141b11cbeafde"
server_key_iv: "844d0635209cfb0c58bc2d45c4c6a1df"
server_key_version: "encryptionkey@0"
I20260929 02:57:54.276836 9045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "3e262df1a9c749629300197f107996a5"
format_stamp: "Formatted at 2026-09-29 02:57:54 on dist-test-slave-m1rv"
server_key: "d3d33028917eeb06923141b11cbeafde"
server_key_iv: "844d0635209cfb0c58bc2d45c4c6a1df"
server_key_version: "encryptionkey@0"
I20260929 02:57:54.282094 9045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260929 02:57:54.285547 9060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:54.286870 9045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:57:54.287060 9045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "3e262df1a9c749629300197f107996a5"
format_stamp: "Formatted at 2026-09-29 02:57:54 on dist-test-slave-m1rv"
server_key: "d3d33028917eeb06923141b11cbeafde"
server_key_iv: "844d0635209cfb0c58bc2d45c4c6a1df"
server_key_version: "encryptionkey@0"
I20260929 02:57:54.287263 9045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:54.330984 9045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:54.336050 9045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:54.336404 9045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:54.356604 9045 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:36009
I20260929 02:57:54.356608 9111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:36009 every 8 connection(s)
I20260929 02:57:54.358217 9045 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:54.362550 9112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:54.366179 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 9045
I20260929 02:57:54.366518 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:57:54.368932 9025 external_mini_cluster.cc:1428] Setting key f9f91a02bb54c12cb81b6b9b369485f4
I20260929 02:57:54.375366 9112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5: Bootstrap starting.
I20260929 02:57:54.378412 9112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:54.379416 9112 log.cc:824] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:54.382197 9112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5: No bootstrap required, opened a new log
I20260929 02:57:54.392360 9112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e262df1a9c749629300197f107996a5" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36009 } }
I20260929 02:57:54.392817 9112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:54.392937 9112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e262df1a9c749629300197f107996a5, State: Initialized, Role: FOLLOWER
I20260929 02:57:54.393635 9112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e262df1a9c749629300197f107996a5" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36009 } }
I20260929 02:57:54.393895 9112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:54.394057 9112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:54.394222 9112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:54.396356 9112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e262df1a9c749629300197f107996a5" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36009 } }
I20260929 02:57:54.396878 9112 leader_election.cc:302] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e262df1a9c749629300197f107996a5; no voters:
I20260929 02:57:54.397492 9112 leader_election.cc:288] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:54.397652 9117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:54.398301 9117 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [term 1 LEADER]: Becoming Leader. State: Replica: 3e262df1a9c749629300197f107996a5, State: Running, Role: LEADER
Sep 29 02:57:54 dist-test-slave-m1rv krb5kdc[9032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:57:54.398985 9117 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e262df1a9c749629300197f107996a5" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36009 } }
I20260929 02:57:54.400409 9112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:54.401033 9119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e262df1a9c749629300197f107996a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e262df1a9c749629300197f107996a5" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36009 } } }
I20260929 02:57:54.401736 9119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:54.402895 9121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:54.403002 9118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e262df1a9c749629300197f107996a5. Latest consensus state: current_term: 1 leader_uuid: "3e262df1a9c749629300197f107996a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e262df1a9c749629300197f107996a5" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36009 } } }
I20260929 02:57:54.403367 9118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:54.413867 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:54.373728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.8.208.126:36009)
0929 02:57:54.374630 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:54.374649 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:54.375925 (+ 1276us) server_negotiation.cc:407] Connection header received
0929 02:57:54.384850 (+ 8925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:54.384875 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:54.385392 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:54.385819 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:54.388325 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:54.390492 (+ 2167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:54.392302 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:54.393369 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:54.393557 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:54.400102 (+ 6545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:54.400114 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:54.400133 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:54.400204 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:54.405478 (+ 5274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:54.406861 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:54.406874 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:54.406883 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:54.407162 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:54.407882 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:54.407887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:54.407890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:54.408640 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:54.411752 (+ 3112us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:54.412020 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 02:57:54.412456 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":394,"threads_started":1}
I20260929 02:57:54.416158 9121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:54.430827 9121 catalog_manager.cc:1347] Generated new cluster ID: 17ee0a3a004f4edbb20d9377838b2389
I20260929 02:57:54.431015 9121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:54.443894 9121 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:54.445654 9121 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:54.452106 9121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e262df1a9c749629300197f107996a5: Generated new TSK 0
I20260929 02:57:54.453107 9121 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 02:57:54.497293 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:36009
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:54.822606 9138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:54.822938 9138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:54.823115 9138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:54.830230 9138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:54.830384 9138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:54.830577 9138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:57:54.840382 9138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:36009
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:54.841715 9138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:54.843044 9138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:54.856518 9143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:54.856498 9144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:54.856552 9146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:54.857556 9138 server_base.cc:1034] running on GCE node
I20260929 02:57:54.858206 9138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:54.859239 9138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:54.860466 9138 hybrid_clock.cc:648] HybridClock initialized: now 1790650674860399 us; error 55 us; skew 500 ppm
I20260929 02:57:54.860860 9138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:54 dist-test-slave-m1rv krb5kdc[9032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:54.864383 9138 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 02:57:54.865996 9138 webserver.cc:460] Webserver started at http://127.8.208.65:39313/ using document root <none> and password file <none>
I20260929 02:57:54.866688 9138 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:54.866823 9138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:54.867139 9138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:54.870034 9138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "679546eec4414f5e88e9d0afff20c540"
format_stamp: "Formatted at 2026-09-29 02:57:54 on dist-test-slave-m1rv"
server_key: "b398295b77a1ade36fc18b48f0585058"
server_key_iv: "27d2e4a019cd48e522dd14f4a7c4952e"
server_key_version: "encryptionkey@0"
I20260929 02:57:54.870863 9138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "679546eec4414f5e88e9d0afff20c540"
format_stamp: "Formatted at 2026-09-29 02:57:54 on dist-test-slave-m1rv"
server_key: "b398295b77a1ade36fc18b48f0585058"
server_key_iv: "27d2e4a019cd48e522dd14f4a7c4952e"
server_key_version: "encryptionkey@0"
I20260929 02:57:54.876143 9138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260929 02:57:54.879524 9153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:54.880834 9138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 02:57:54.880997 9138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "679546eec4414f5e88e9d0afff20c540"
format_stamp: "Formatted at 2026-09-29 02:57:54 on dist-test-slave-m1rv"
server_key: "b398295b77a1ade36fc18b48f0585058"
server_key_iv: "27d2e4a019cd48e522dd14f4a7c4952e"
server_key_version: "encryptionkey@0"
I20260929 02:57:54.881247 9138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:54.892323 9138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:54.896566 9138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:54.896878 9138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:54.897941 9138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:57:54.899296 9138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:57:54.899375 9138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:54.899528 9138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:57:54.899600 9138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:54.935510 9138 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:41759
I20260929 02:57:54.935554 9265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:41759 every 8 connection(s)
I20260929 02:57:54.936883 9138 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:57:54.943881 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 9138
I20260929 02:57:54.944113 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:57:54.944890 9025 external_mini_cluster.cc:1428] Setting key 99b203715d8b87c945eba162da727a72
Sep 29 02:57:54 dist-test-slave-m1rv krb5kdc[9032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
I20260929 02:57:54.959612 9268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:54.939437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:59485 (local address 127.8.208.126:36009)
0929 02:57:54.939862 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:54.939867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:54.940937 (+ 1070us) server_negotiation.cc:407] Connection header received
0929 02:57:54.942821 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:54.942827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:54.942918 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:54.943036 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:54.945105 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:54.946116 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:54.947823 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:54.948259 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:54.948409 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:54.952676 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:54.952683 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:54.952687 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:54.952722 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:54.955852 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:54.956738 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:54.956747 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:54.956751 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:54.956902 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:54.957497 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:54.957505 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:54.957510 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:54.957832 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:54.958020 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:54.958837 (+ 817us) server_negotiation.cc:299] Negotiation successful
0929 02:57:54.959106 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":229,"threads_started":1}
I20260929 02:57:54.960784 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:54.940079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:36009 (local address 127.8.208.65:59485)
0929 02:57:54.940773 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:54.940810 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:54.942547 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:54.943265 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:54.943272 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:54.944029 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:54.944868 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:54.944910 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:54.946291 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:54.946298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:54.947636 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:54.947654 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:54.948489 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:54.948497 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:54.948771 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:54.949535 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:54.949612 (+ 77us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:54.952397 (+ 2785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:54.956030 (+ 3633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:54.956041 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:54.956060 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:54.956590 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:54.957040 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:54.957047 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:54.957052 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:54.957365 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:54.957996 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:54.958005 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:54.958410 (+ 405us) client_negotiation.cc:769] Sending connection context
0929 02:57:54.958733 (+ 323us) client_negotiation.cc:240] Negotiation successful
0929 02:57:54.959357 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":270,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
I20260929 02:57:54.962534 9266 heartbeater.cc:347] Connected to a master server at 127.8.208.126:36009
I20260929 02:57:54.962857 9266 heartbeater.cc:464] Registering TS with master...
I20260929 02:57:54.963605 9266 heartbeater.cc:511] Master 127.8.208.126:36009 requested a full tablet report, sending...
I20260929 02:57:54.965710 9077 ts_manager.cc:194] Registered new tserver with Master: 679546eec4414f5e88e9d0afff20c540 (127.8.208.65:41759)
I20260929 02:57:54.967640 9077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:59485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 02:57:54.983990 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:36009
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:55.302937 9272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:55.303222 9272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:55.303395 9272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:55.310505 9272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:55.310611 9272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:55.310845 9272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:57:55.320567 9272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:36009
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:55.321898 9272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:55.323240 9272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:55.336215 9278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:55.336340 9277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:55.336266 9280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:55.336980 9272 server_base.cc:1034] running on GCE node
I20260929 02:57:55.337671 9272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:55.338546 9272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:55.339769 9272 hybrid_clock.cc:648] HybridClock initialized: now 1790650675339723 us; error 58 us; skew 500 ppm
I20260929 02:57:55.340080 9272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:55 dist-test-slave-m1rv krb5kdc[9032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650675, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:55.343211 9272 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 02:57:55.344638 9272 webserver.cc:460] Webserver started at http://127.8.208.66:35215/ using document root <none> and password file <none>
I20260929 02:57:55.345366 9272 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:55.345512 9272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:55.345825 9272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:55.348683 9272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "ade9e016abf74d94ab79e4e24457b97f"
format_stamp: "Formatted at 2026-09-29 02:57:55 on dist-test-slave-m1rv"
server_key: "4845f402c031f852c80d8e1f21d26270"
server_key_iv: "d1a7a5154c777ccca85f373b2a1854f2"
server_key_version: "encryptionkey@0"
I20260929 02:57:55.349505 9272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "ade9e016abf74d94ab79e4e24457b97f"
format_stamp: "Formatted at 2026-09-29 02:57:55 on dist-test-slave-m1rv"
server_key: "4845f402c031f852c80d8e1f21d26270"
server_key_iv: "d1a7a5154c777ccca85f373b2a1854f2"
server_key_version: "encryptionkey@0"
I20260929 02:57:55.354390 9272 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260929 02:57:55.357882 9287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:55.359180 9272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:57:55.359341 9272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "ade9e016abf74d94ab79e4e24457b97f"
format_stamp: "Formatted at 2026-09-29 02:57:55 on dist-test-slave-m1rv"
server_key: "4845f402c031f852c80d8e1f21d26270"
server_key_iv: "d1a7a5154c777ccca85f373b2a1854f2"
server_key_version: "encryptionkey@0"
I20260929 02:57:55.359539 9272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:55.397259 9272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:55.401507 9272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:55.401818 9272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:55.402973 9272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:57:55.404541 9272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:57:55.404626 9272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:55.404711 9272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:57:55.404753 9272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:55.438254 9272 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:32875
I20260929 02:57:55.438302 9399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:32875 every 8 connection(s)
I20260929 02:57:55.439582 9272 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:57:55.449517 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 9272
I20260929 02:57:55.449729 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:57:55.450148 9025 external_mini_cluster.cc:1428] Setting key 626fde28ea1bd278e227a4350bf8485a
Sep 29 02:57:55 dist-test-slave-m1rv krb5kdc[9032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650675, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:57:55.461684 9268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:55.442398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:49581 (local address 127.8.208.126:36009)
0929 02:57:55.442614 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:55.442618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:55.443631 (+ 1013us) server_negotiation.cc:407] Connection header received
0929 02:57:55.445471 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:55.445477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:55.445562 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:55.445666 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:55.448021 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:55.448785 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:55.450675 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:55.451254 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:55.451392 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:55.455532 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:55.455537 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:55.455540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:55.455573 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:55.458494 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:55.459295 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:55.459301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:55.459306 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:55.459443 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:55.459897 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:55.459906 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:55.459908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:55.460207 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:55.460365 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:55.461076 (+ 711us) server_negotiation.cc:299] Negotiation successful
0929 02:57:55.461345 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260929 02:57:55.462842 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:55.442814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:36009 (local address 127.8.208.66:49581)
0929 02:57:55.443471 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:55.443507 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:55.445095 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:55.446099 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:55.446108 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:55.446945 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:55.447832 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:55.447858 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:55.448938 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:55.448943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:55.450448 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:55.450459 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:55.451484 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:55.451490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:55.451718 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:55.452626 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:55.452656 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:55.455302 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:55.458682 (+ 3380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:55.458692 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:55.458712 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:55.459159 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:55.459561 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:55.459566 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:55.459569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:55.459772 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:55.460340 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:55.460348 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:55.460713 (+ 365us) client_negotiation.cc:769] Sending connection context
0929 02:57:55.460979 (+ 266us) client_negotiation.cc:240] Negotiation successful
0929 02:57:55.461561 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":256,"threads_started":1}
I20260929 02:57:55.464455 9400 heartbeater.cc:347] Connected to a master server at 127.8.208.126:36009
I20260929 02:57:55.464800 9400 heartbeater.cc:464] Registering TS with master...
I20260929 02:57:55.465569 9400 heartbeater.cc:511] Master 127.8.208.126:36009 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
I20260929 02:57:55.467092 9077 ts_manager.cc:194] Registered new tserver with Master: ade9e016abf74d94ab79e4e24457b97f (127.8.208.66:32875)
I20260929 02:57:55.467878 9077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:49581
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 02:57:55.488579 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:36009
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:55.804541 9405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:55.804823 9405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:55.804988 9405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:55.812156 9405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:55.812321 9405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:55.812522 9405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:57:55.822393 9405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:36009
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:55.823673 9405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:55.825023 9405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:55.838112 9410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:55.838132 9411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:55.838168 9413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:55.839010 9405 server_base.cc:1034] running on GCE node
I20260929 02:57:55.839568 9405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:55.840487 9405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:55.841720 9405 hybrid_clock.cc:648] HybridClock initialized: now 1790650675841670 us; error 60 us; skew 500 ppm
I20260929 02:57:55.842037 9405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:55 dist-test-slave-m1rv krb5kdc[9032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650675, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:55.845399 9405 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 02:57:55.846756 9405 webserver.cc:460] Webserver started at http://127.8.208.67:35097/ using document root <none> and password file <none>
I20260929 02:57:55.847411 9405 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:55.847512 9405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:55.847771 9405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:55.850960 9405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "f08dfb84f8914461a87d951ebd557468"
format_stamp: "Formatted at 2026-09-29 02:57:55 on dist-test-slave-m1rv"
server_key: "02e6bc1f204b7e2c42917a7f20d5368a"
server_key_iv: "6f48c29c8fc1e8800dccbb10bf6e9897"
server_key_version: "encryptionkey@0"
I20260929 02:57:55.851861 9405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "f08dfb84f8914461a87d951ebd557468"
format_stamp: "Formatted at 2026-09-29 02:57:55 on dist-test-slave-m1rv"
server_key: "02e6bc1f204b7e2c42917a7f20d5368a"
server_key_iv: "6f48c29c8fc1e8800dccbb10bf6e9897"
server_key_version: "encryptionkey@0"
I20260929 02:57:55.857498 9405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 02:57:55.860929 9420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:55.862458 9405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:57:55.862641 9405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "f08dfb84f8914461a87d951ebd557468"
format_stamp: "Formatted at 2026-09-29 02:57:55 on dist-test-slave-m1rv"
server_key: "02e6bc1f204b7e2c42917a7f20d5368a"
server_key_iv: "6f48c29c8fc1e8800dccbb10bf6e9897"
server_key_version: "encryptionkey@0"
I20260929 02:57:55.862812 9405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:55.882262 9405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:55.887008 9405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:55.887259 9405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:55.888156 9405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:57:55.889528 9405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:57:55.889601 9405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:55.889694 9405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:57:55.889750 9405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:55.922782 9405 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:43873
I20260929 02:57:55.922833 9532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:43873 every 8 connection(s)
I20260929 02:57:55.924043 9405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:57:55.929932 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 9405
I20260929 02:57:55.930121 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:57:55.930553 9025 external_mini_cluster.cc:1428] Setting key 28cc96350a61540668bb50550aff1ca0
Sep 29 02:57:55 dist-test-slave-m1rv krb5kdc[9032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650675, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:57:55.944605 9268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:55.926491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:58093 (local address 127.8.208.126:36009)
0929 02:57:55.926674 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:55.926678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:55.927754 (+ 1076us) server_negotiation.cc:407] Connection header received
0929 02:57:55.929544 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:55.929549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:55.929677 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:55.929787 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:55.931762 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:55.932885 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:55.934739 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:55.935332 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:55.935457 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:55.939099 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:55.939106 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:55.939111 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:55.939151 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:55.942007 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:55.942697 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:55.942701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:55.942703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:55.942820 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:55.943189 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:55.943192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:55.943194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:55.943445 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:55.943548 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:55.944139 (+ 591us) server_negotiation.cc:299] Negotiation successful
0929 02:57:55.944295 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:57:55.945709 9535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:55.926901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:36009 (local address 127.8.208.67:58093)
0929 02:57:55.927591 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:55.927627 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:55.929274 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:55.930069 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:55.930085 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:55.930692 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:55.931589 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:55.931610 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:55.933073 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:55.933080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:55.934605 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:55.934617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:55.935532 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:55.935537 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:55.935730 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:55.936453 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:55.936475 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:55.938870 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:55.942148 (+ 3278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:55.942155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:55.942167 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:55.942587 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:55.942918 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:55.942921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:55.942923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:55.943095 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:55.943542 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:55.943547 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:55.943840 (+ 293us) client_negotiation.cc:769] Sending connection context
0929 02:57:55.944054 (+ 214us) client_negotiation.cc:240] Negotiation successful
0929 02:57:55.944440 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":262,"threads_started":1}
I20260929 02:57:55.947100 9533 heartbeater.cc:347] Connected to a master server at 127.8.208.126:36009
I20260929 02:57:55.947396 9533 heartbeater.cc:464] Registering TS with master...
I20260929 02:57:55.948081 9533 heartbeater.cc:511] Master 127.8.208.126:36009 requested a full tablet report, sending...
I20260929 02:57:55.949522 9076 ts_manager.cc:194] Registered new tserver with Master: f08dfb84f8914461a87d951ebd557468 (127.8.208.67:43873)
I20260929 02:57:55.950322 9076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:58093
I20260929 02:57:55.959131 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:57:55.970948 9266 heartbeater.cc:503] Master 127.8.208.126:36009 was elected leader, sending a full tablet report...
Sep 29 02:57:55 dist-test-slave-m1rv krb5kdc[9032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650675, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:57:55 dist-test-slave-m1rv krb5kdc[9032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650675, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:57:56.004491 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:55.988803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56098 (local address 127.8.208.65:41759)
0929 02:57:55.989494 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:55.989505 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:55.989607 (+ 102us) server_negotiation.cc:407] Connection header received
0929 02:57:55.989827 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:55.989844 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:55.990074 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:55.990243 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:55.991726 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:55.992735 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:55.994069 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:55.994639 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:55.994778 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:55.998981 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:55.998993 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:55.999007 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:55.999051 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:56.002060 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:56.002731 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:56.002737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:56.002739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:56.002858 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:56.003201 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:56.003205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:56.003207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:56.003543 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:56.003751 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:56.003997 (+ 246us) server_negotiation.cc:299] Negotiation successful
0929 02:57:56.004190 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":356,"threads_started":1}
W20260929 02:57:56.005829 9181 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56098
Sep 29 02:57:56 dist-test-slave-m1rv krb5kdc[9032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:57:56 dist-test-slave-m1rv krb5kdc[9032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:57:56.053817 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:56.035028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56102 (local address 127.8.208.65:41759)
0929 02:57:56.035317 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:56.035325 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:56.035589 (+ 264us) server_negotiation.cc:407] Connection header received
0929 02:57:56.035835 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:56.035840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:56.035909 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:56.036015 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:56.037426 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:56.038104 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:56.039352 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:56.039776 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:56.039913 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:56.048675 (+ 8762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:56.048680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:56.048683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:56.048718 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:56.051422 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:56.052243 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:56.052251 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:56.052255 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:56.052413 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:56.052708 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:56.052711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:56.052713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:56.052949 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:56.053114 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:56.053427 (+ 313us) server_negotiation.cc:299] Negotiation successful
0929 02:57:56.053585 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 02:57:56.055264 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 9138
I20260929 02:57:56.068135 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:56.056284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56116 (local address 127.8.208.65:41759)
0929 02:57:56.056516 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:56.056522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:56.056726 (+ 204us) server_negotiation.cc:407] Connection header received
0929 02:57:56.056907 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:56.056912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:56.056974 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:56.057074 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:56.058223 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:56.058908 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:56.060133 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:56.060549 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:56.060686 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:56.063094 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:56.063098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:56.063102 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:56.063130 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:56.065630 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:56.066240 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:56.066248 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:56.066251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:56.066412 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:56.066783 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:56.066792 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:56.066794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:56.067025 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:56.067266 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:56.067573 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 02:57:56.067799 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:57:56.068955 9261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:56.152808 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 9138
I20260929 02:57:56.169252 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 9272
Sep 29 02:57:56 dist-test-slave-m1rv krb5kdc[9032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 02:57:56.184640 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:56.170142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41720 (local address 127.8.208.66:32875)
0929 02:57:56.170639 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:56.170646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:56.170669 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:57:56.170726 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:56.170732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:56.170936 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:56.171059 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:56.172259 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:56.173306 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:56.174503 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:56.175021 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:56.175114 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:56.179107 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:56.179119 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:56.179134 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:56.179176 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:56.182151 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:56.182763 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:56.182770 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:56.182773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:56.182919 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:56.183241 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:56.183244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:56.183246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:56.183568 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:56.183779 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:56.184055 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 02:57:56.184248 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":255,"threads_started":1}
I20260929 02:57:56.186053 9395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:56.267930 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 9272
I20260929 02:57:56.283524 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 9405
Sep 29 02:57:56 dist-test-slave-m1rv krb5kdc[9032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 02:57:56.299093 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:56.284340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55582 (local address 127.8.208.67:43873)
0929 02:57:56.284900 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:56.284907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:56.284934 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:57:56.284991 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:56.284997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:56.285265 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:56.285428 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:56.286676 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:56.287673 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:56.288936 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:56.289606 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:56.289748 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:56.293796 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:56.293806 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:56.293821 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:56.293859 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:56.296369 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:56.297002 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:56.297015 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:56.297017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:56.297126 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:56.297536 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:56.297542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:56.297546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:56.297880 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:56.298184 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:56.298482 (+ 298us) server_negotiation.cc:299] Negotiation successful
0929 02:57:56.298744 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":266,"threads_started":1}
I20260929 02:57:56.300359 9528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:56.382951 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 9405
I20260929 02:57:56.398355 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 9045
I20260929 02:57:56.399484 9107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:57:56.509265 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 9045
2026-09-29T02:57:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3894 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:57:56 dist-test-slave-m1rv krb5kdc[9554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:57:56 dist-test-slave-m1rv krb5kdc[9554](info): set up 2 sockets
Sep 29 02:57:56 dist-test-slave-m1rv krb5kdc[9554](info): commencing operation
krb5kdc: starting...
W20260929 02:57:57.585049 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.002s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:57:57 dist-test-slave-m1rv krb5kdc[9554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:57:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:57:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:57:57.756932 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:38755
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:38755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:58.071770 9567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:58.072055 9567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:58.072141 9567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:58.079231 9567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:57:58.079339 9567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:58.079384 9567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:58.079442 9567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:57:58.079478 9567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:57:58.089125 9567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:38755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:38755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:58.090337 9567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:58.091519 9567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:58.103183 9572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:58.103123 9575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:58.103096 9573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:58.103874 9567 server_base.cc:1034] running on GCE node
I20260929 02:57:58.104648 9567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:58.106208 9567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:58.107440 9567 hybrid_clock.cc:648] HybridClock initialized: now 1790650678107370 us; error 59 us; skew 500 ppm
I20260929 02:57:58.107779 9567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:58 dist-test-slave-m1rv krb5kdc[9554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:58.110917 9567 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:57:58.112336 9567 webserver.cc:460] Webserver started at http://127.8.208.126:36085/ using document root <none> and password file <none>
I20260929 02:57:58.112996 9567 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:58.113092 9567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:58.113411 9567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:58.116169 9567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "5335db50dce2482e8eb7fb14a129a20a"
format_stamp: "Formatted at 2026-09-29 02:57:58 on dist-test-slave-m1rv"
server_key: "c4d92c708339263de1ab91d49187b3d1"
server_key_iv: "56466b9062c1b956190f4601946ea11f"
server_key_version: "encryptionkey@0"
I20260929 02:57:58.116854 9567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "5335db50dce2482e8eb7fb14a129a20a"
format_stamp: "Formatted at 2026-09-29 02:57:58 on dist-test-slave-m1rv"
server_key: "c4d92c708339263de1ab91d49187b3d1"
server_key_iv: "56466b9062c1b956190f4601946ea11f"
server_key_version: "encryptionkey@0"
I20260929 02:57:58.121687 9567 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260929 02:57:58.125006 9582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:58.126299 9567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:57:58.126492 9567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "5335db50dce2482e8eb7fb14a129a20a"
format_stamp: "Formatted at 2026-09-29 02:57:58 on dist-test-slave-m1rv"
server_key: "c4d92c708339263de1ab91d49187b3d1"
server_key_iv: "56466b9062c1b956190f4601946ea11f"
server_key_version: "encryptionkey@0"
I20260929 02:57:58.126682 9567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:58.149907 9567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:58.153791 9567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:58.154080 9567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:58.171828 9567 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:38755
I20260929 02:57:58.171836 9633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:38755 every 8 connection(s)
I20260929 02:57:58.173228 9567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:57:58.176833 9634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:58.182632 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 9567
I20260929 02:57:58.182862 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:57:58.183504 9025 external_mini_cluster.cc:1428] Setting key eef3065aa9130c17cb81bbfebbad99fb
I20260929 02:57:58.187398 9634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a: Bootstrap starting.
I20260929 02:57:58.191150 9634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:58.192396 9634 log.cc:824] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:58.195825 9634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a: No bootstrap required, opened a new log
Sep 29 02:57:58 dist-test-slave-m1rv krb5kdc[9554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:57:58.205130 9634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5335db50dce2482e8eb7fb14a129a20a" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 38755 } }
I20260929 02:57:58.205644 9634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:58.205756 9634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5335db50dce2482e8eb7fb14a129a20a, State: Initialized, Role: FOLLOWER
I20260929 02:57:58.206451 9634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5335db50dce2482e8eb7fb14a129a20a" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 38755 } }
I20260929 02:57:58.206729 9634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:57:58.206871 9634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:57:58.207054 9634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:57:58.208073 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:58.185441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38314 (local address 127.8.208.126:38755)
0929 02:57:58.186229 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:58.186243 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:58.186293 (+ 50us) server_negotiation.cc:407] Connection header received
0929 02:57:58.188558 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:58.188584 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:58.189083 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:58.189546 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:58.190743 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:58.192747 (+ 2004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:58.194320 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:58.195305 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:58.195482 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:58.199418 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:58.199428 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:58.199447 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:58.199492 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:58.203523 (+ 4031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:58.204200 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:58.204208 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:58.204217 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:58.204426 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:58.204747 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:58.204752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:58.204755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:58.205437 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:58.205743 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:58.206260 (+ 517us) server_negotiation.cc:299] Negotiation successful
0929 02:57:58.206689 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":322,"threads_started":1}
I20260929 02:57:58.209774 9634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5335db50dce2482e8eb7fb14a129a20a" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 38755 } }
I20260929 02:57:58.210261 9634 leader_election.cc:302] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5335db50dce2482e8eb7fb14a129a20a; no voters:
I20260929 02:57:58.210872 9634 leader_election.cc:288] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:57:58.210992 9639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:57:58.211676 9639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [term 1 LEADER]: Becoming Leader. State: Replica: 5335db50dce2482e8eb7fb14a129a20a, State: Running, Role: LEADER
I20260929 02:57:58.212176 9639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5335db50dce2482e8eb7fb14a129a20a" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 38755 } }
I20260929 02:57:58.212849 9634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:57:58.214943 9640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5335db50dce2482e8eb7fb14a129a20a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5335db50dce2482e8eb7fb14a129a20a" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 38755 } } }
I20260929 02:57:58.215284 9640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:58.215994 9641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5335db50dce2482e8eb7fb14a129a20a. Latest consensus state: current_term: 1 leader_uuid: "5335db50dce2482e8eb7fb14a129a20a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5335db50dce2482e8eb7fb14a129a20a" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 38755 } } }
I20260929 02:57:58.216259 9641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a [sys.catalog]: This master's current role is: LEADER
I20260929 02:57:58.216936 9645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:57:58.221877 9645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:57:58.229615 9645 catalog_manager.cc:1347] Generated new cluster ID: ca45d794b8474d8596eb5aaf611c74ef
I20260929 02:57:58.229730 9645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:57:58.266009 9645 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:57:58.267875 9645 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:57:58.276467 9645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5335db50dce2482e8eb7fb14a129a20a: Generated new TSK 0
I20260929 02:57:58.277465 9645 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 02:57:58.324958 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:38755
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:58.646175 9660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:58.646456 9660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:58.646620 9660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:58.653986 9660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:58.654129 9660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:58.654325 9660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:57:58.667214 9660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:38755
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:58.668586 9660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:58.670030 9660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:58.683408 9666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:58.683532 9668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:58.684129 9665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:58.684204 9660 server_base.cc:1034] running on GCE node
I20260929 02:57:58.684968 9660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:58.686218 9660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:58.687500 9660 hybrid_clock.cc:648] HybridClock initialized: now 1790650678687434 us; error 52 us; skew 500 ppm
I20260929 02:57:58.687896 9660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:58 dist-test-slave-m1rv krb5kdc[9554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:58.691843 9660 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 02:57:58.693404 9660 webserver.cc:460] Webserver started at http://127.8.208.65:42439/ using document root <none> and password file <none>
I20260929 02:57:58.694170 9660 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:58.694314 9660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:58.694638 9660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:58.697669 9660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "db729f9d2ba24ef484077504795746da"
format_stamp: "Formatted at 2026-09-29 02:57:58 on dist-test-slave-m1rv"
server_key: "0a680151dd4d6450bfc4aec91bf60603"
server_key_iv: "48abb414c695007e7ba03c1f03bf7e95"
server_key_version: "encryptionkey@0"
I20260929 02:57:58.698477 9660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "db729f9d2ba24ef484077504795746da"
format_stamp: "Formatted at 2026-09-29 02:57:58 on dist-test-slave-m1rv"
server_key: "0a680151dd4d6450bfc4aec91bf60603"
server_key_iv: "48abb414c695007e7ba03c1f03bf7e95"
server_key_version: "encryptionkey@0"
I20260929 02:57:58.706988 9660 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260929 02:57:58.711447 9675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:58.713233 9660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:57:58.713436 9660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "db729f9d2ba24ef484077504795746da"
format_stamp: "Formatted at 2026-09-29 02:57:58 on dist-test-slave-m1rv"
server_key: "0a680151dd4d6450bfc4aec91bf60603"
server_key_iv: "48abb414c695007e7ba03c1f03bf7e95"
server_key_version: "encryptionkey@0"
I20260929 02:57:58.713649 9660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:58.726914 9660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:58.731508 9660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:58.731815 9660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:58.732785 9660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:57:58.734139 9660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:57:58.734246 9660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:58.734418 9660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:57:58.734493 9660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:58.767899 9660 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:42545
I20260929 02:57:58.767943 9787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:42545 every 8 connection(s)
I20260929 02:57:58.769243 9660 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:57:58.770892 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 9660
I20260929 02:57:58.771111 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:57:58.771764 9025 external_mini_cluster.cc:1428] Setting key 20422b7bf7674e7a95ee84e331dc2c29
Sep 29 02:57:58 dist-test-slave-m1rv krb5kdc[9554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
I20260929 02:57:58.789680 9790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:58.771988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:36995 (local address 127.8.208.126:38755)
0929 02:57:58.772505 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:58.772514 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:58.773629 (+ 1115us) server_negotiation.cc:407] Connection header received
0929 02:57:58.775514 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:58.775520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:58.775602 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:58.775741 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:58.777577 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:58.778296 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:58.779844 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:58.780258 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:58.780401 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:58.783853 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:58.783862 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:58.783872 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:58.783906 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:58.786576 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:58.787348 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:58.787353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:58.787355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:58.787477 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:58.787922 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:58.787928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:58.787936 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:58.788199 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:58.788357 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:58.789068 (+ 711us) server_negotiation.cc:299] Negotiation successful
0929 02:57:58.789312 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":240,"threads_started":1}
I20260929 02:57:58.790701 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:58.772595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:38755 (local address 127.8.208.65:36995)
0929 02:57:58.773469 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:58.773514 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:58.775220 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:58.776035 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:58.776047 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:58.776726 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:58.777393 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:58.777413 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:58.778473 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:58.778478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:58.779699 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:58.779711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:58.780490 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:58.780497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:58.780726 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:58.781472 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:58.781495 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:58.783622 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:58.786717 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:58.786726 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:58.786743 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:58.787221 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:58.787599 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:58.787604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:58.787608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:58.787795 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:58.788331 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:58.788339 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:58.788682 (+ 343us) client_negotiation.cc:769] Sending connection context
0929 02:57:58.788955 (+ 273us) client_negotiation.cc:240] Negotiation successful
0929 02:57:58.789521 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":378,"threads_started":1}
I20260929 02:57:58.792306 9788 heartbeater.cc:347] Connected to a master server at 127.8.208.126:38755
I20260929 02:57:58.792654 9788 heartbeater.cc:464] Registering TS with master...
I20260929 02:57:58.793421 9788 heartbeater.cc:511] Master 127.8.208.126:38755 requested a full tablet report, sending...
I20260929 02:57:58.795727 9599 ts_manager.cc:194] Registered new tserver with Master: db729f9d2ba24ef484077504795746da (127.8.208.65:42545)
I20260929 02:57:58.797951 9599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:36995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 02:57:58.813112 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:38755
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:59.129338 9794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:59.129617 9794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:59.129774 9794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:59.137105 9794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:59.137233 9794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:59.137413 9794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:57:59.146883 9794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:38755
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:59.148140 9794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:59.149394 9794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:59.162169 9799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:59.162138 9800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:59.162109 9802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:59.163009 9794 server_base.cc:1034] running on GCE node
I20260929 02:57:59.163637 9794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:59.164590 9794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:59.165820 9794 hybrid_clock.cc:648] HybridClock initialized: now 1790650679165777 us; error 69 us; skew 500 ppm
I20260929 02:57:59.166134 9794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:59 dist-test-slave-m1rv krb5kdc[9554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650679, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:59.169356 9794 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 02:57:59.170779 9794 webserver.cc:460] Webserver started at http://127.8.208.66:35893/ using document root <none> and password file <none>
I20260929 02:57:59.171546 9794 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:59.171684 9794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:59.171989 9794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:59.174816 9794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "e6e0a104380a4d07acbf6e949fafcba7"
format_stamp: "Formatted at 2026-09-29 02:57:59 on dist-test-slave-m1rv"
server_key: "bb36ca7a42771e94adaeb112d94e36fc"
server_key_iv: "b126544d193f0f03f0373da2cbcb0bea"
server_key_version: "encryptionkey@0"
I20260929 02:57:59.175575 9794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "e6e0a104380a4d07acbf6e949fafcba7"
format_stamp: "Formatted at 2026-09-29 02:57:59 on dist-test-slave-m1rv"
server_key: "bb36ca7a42771e94adaeb112d94e36fc"
server_key_iv: "b126544d193f0f03f0373da2cbcb0bea"
server_key_version: "encryptionkey@0"
I20260929 02:57:59.180658 9794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:57:59.184388 9809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:59.185870 9794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:57:59.186074 9794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "e6e0a104380a4d07acbf6e949fafcba7"
format_stamp: "Formatted at 2026-09-29 02:57:59 on dist-test-slave-m1rv"
server_key: "bb36ca7a42771e94adaeb112d94e36fc"
server_key_iv: "b126544d193f0f03f0373da2cbcb0bea"
server_key_version: "encryptionkey@0"
I20260929 02:57:59.186272 9794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:59.230737 9794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:59.235215 9794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:59.235525 9794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:59.236569 9794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:57:59.237922 9794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:57:59.238024 9794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:59.238178 9794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:57:59.238252 9794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:59.271014 9794 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:43687
I20260929 02:57:59.271034 9921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:43687 every 8 connection(s)
I20260929 02:57:59.272295 9794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:57:59.279870 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 9794
I20260929 02:57:59.280159 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:57:59.280596 9025 external_mini_cluster.cc:1428] Setting key 911ce050685d34be87849b38f3641cd6
Sep 29 02:57:59 dist-test-slave-m1rv krb5kdc[9554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650679, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:57:59.292810 9790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.274859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:55961 (local address 127.8.208.126:38755)
0929 02:57:59.275044 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:59.275049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:59.276104 (+ 1055us) server_negotiation.cc:407] Connection header received
0929 02:57:59.278188 (+ 2084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:59.278194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:59.278290 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:59.278413 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:59.279870 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.280727 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.282522 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.283134 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.283269 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.287034 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:59.287039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:59.287042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:59.287071 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:59.289841 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:59.290599 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:59.290603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:59.290606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:59.290745 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:59.291169 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:59.291178 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:59.291180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:59.291435 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:59.291602 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:59.292240 (+ 638us) server_negotiation.cc:299] Negotiation successful
0929 02:57:59.292468 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 02:57:59.293883 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.275282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:38755 (local address 127.8.208.66:55961)
0929 02:57:59.275938 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:59.275981 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:59.277919 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:59.278642 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:59.278650 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:59.279151 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:59.279702 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.279718 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.280871 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.280881 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.282369 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.282384 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.283342 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.283347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.283538 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.284228 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:59.284250 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:59.286807 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:59.289972 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:59.289980 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:59.290009 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:59.290475 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:59.290861 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:59.290865 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:59.290869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:59.291050 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:59.291557 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:59.291565 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:59.291901 (+ 336us) client_negotiation.cc:769] Sending connection context
0929 02:57:59.292150 (+ 249us) client_negotiation.cc:240] Negotiation successful
0929 02:57:59.292664 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":249,"threads_started":1}
I20260929 02:57:59.295270 9922 heartbeater.cc:347] Connected to a master server at 127.8.208.126:38755
I20260929 02:57:59.295568 9922 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
I20260929 02:57:59.296216 9922 heartbeater.cc:511] Master 127.8.208.126:38755 requested a full tablet report, sending...
I20260929 02:57:59.297693 9599 ts_manager.cc:194] Registered new tserver with Master: e6e0a104380a4d07acbf6e949fafcba7 (127.8.208.66:43687)
I20260929 02:57:59.298563 9599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:55961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 02:57:59.318943 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:38755
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:57:59.641930 9927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:57:59.642202 9927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:57:59.642351 9927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:57:59.649245 9927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:57:59.649358 9927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:57:59.649479 9927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:57:59.658928 9927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:38755
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:57:59.660176 9927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:57:59.661387 9927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:57:59.674316 9933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:59.674324 9935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:57:59.674324 9932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:57:59.674954 9927 server_base.cc:1034] running on GCE node
I20260929 02:57:59.675546 9927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:57:59.676470 9927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:57:59.677687 9927 hybrid_clock.cc:648] HybridClock initialized: now 1790650679677647 us; error 57 us; skew 500 ppm
I20260929 02:57:59.677997 9927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:57:59 dist-test-slave-m1rv krb5kdc[9554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650679, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:57:59.681067 9927 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 02:57:59.682505 9927 webserver.cc:460] Webserver started at http://127.8.208.67:36477/ using document root <none> and password file <none>
I20260929 02:57:59.683213 9927 fs_manager.cc:359] Metadata directory not provided
I20260929 02:57:59.683352 9927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:57:59.683655 9927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:57:59.686523 9927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "def5352312a04934a14f0bbf0dad01d8"
format_stamp: "Formatted at 2026-09-29 02:57:59 on dist-test-slave-m1rv"
server_key: "fba6876e32f8b5c4faabfd7363c23c20"
server_key_iv: "240dde0fabb44dc12717ad2854e1544c"
server_key_version: "encryptionkey@0"
I20260929 02:57:59.687278 9927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "def5352312a04934a14f0bbf0dad01d8"
format_stamp: "Formatted at 2026-09-29 02:57:59 on dist-test-slave-m1rv"
server_key: "fba6876e32f8b5c4faabfd7363c23c20"
server_key_iv: "240dde0fabb44dc12717ad2854e1544c"
server_key_version: "encryptionkey@0"
I20260929 02:57:59.692438 9927 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:57:59.695782 9942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:59.696987 9927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:57:59.697157 9927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "def5352312a04934a14f0bbf0dad01d8"
format_stamp: "Formatted at 2026-09-29 02:57:59 on dist-test-slave-m1rv"
server_key: "fba6876e32f8b5c4faabfd7363c23c20"
server_key_iv: "240dde0fabb44dc12717ad2854e1544c"
server_key_version: "encryptionkey@0"
I20260929 02:57:59.697423 9927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:57:59.712252 9927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:57:59.716312 9927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:57:59.716647 9927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:57:59.717767 9927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:57:59.719125 9927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:57:59.719223 9927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:59.719348 9927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:57:59.719398 9927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:57:59.750380 9927 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:33969
I20260929 02:57:59.750427 10054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:33969 every 8 connection(s)
I20260929 02:57:59.751662 9927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:57:59.754350 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 9927
I20260929 02:57:59.754546 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:57:59.754971 9025 external_mini_cluster.cc:1428] Setting key d18cad4418d29feed081d75949e8160a
Sep 29 02:57:59 dist-test-slave-m1rv krb5kdc[9554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650679, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:57:59.771960 9790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.754118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:46401 (local address 127.8.208.126:38755)
0929 02:57:59.754358 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:59.754365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:59.755625 (+ 1260us) server_negotiation.cc:407] Connection header received
0929 02:57:59.757976 (+ 2351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:59.757983 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:59.758068 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:59.758147 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:59.759538 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.760446 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.761989 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.762378 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.762474 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.765752 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:59.765757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:59.765760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:59.765790 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:59.768695 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:59.769608 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:59.769611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:59.769614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:59.769761 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:59.770273 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:59.770278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:59.770281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:59.770570 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:59.770679 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:59.771553 (+ 874us) server_negotiation.cc:299] Negotiation successful
0929 02:57:59.771711 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 02:57:59.773280 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.754616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:38755 (local address 127.8.208.67:46401)
0929 02:57:59.755445 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:59.755489 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:59.757645 (+ 2156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:59.758347 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:59.758369 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:59.758846 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:57:59.759400 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.759415 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.760575 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.760578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.761868 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.761879 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.762557 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.762562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.762754 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.763439 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:57:59.763461 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:57:59.765566 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:57:59.768828 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:59.768841 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:59.768858 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:59.769461 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:59.769883 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:57:59.769897 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:57:59.769901 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:57:59.770144 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:57:59.770702 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:57:59.770711 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:57:59.771130 (+ 419us) client_negotiation.cc:769] Sending connection context
0929 02:57:59.771417 (+ 287us) client_negotiation.cc:240] Negotiation successful
0929 02:57:59.771949 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":333,"threads_started":1}
I20260929 02:57:59.774646 10055 heartbeater.cc:347] Connected to a master server at 127.8.208.126:38755
I20260929 02:57:59.774951 10055 heartbeater.cc:464] Registering TS with master...
I20260929 02:57:59.775638 10055 heartbeater.cc:511] Master 127.8.208.126:38755 requested a full tablet report, sending...
I20260929 02:57:59.776998 9599 ts_manager.cc:194] Registered new tserver with Master: def5352312a04934a14f0bbf0dad01d8 (127.8.208.67:33969)
I20260929 02:57:59.777841 9599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:46401
I20260929 02:57:59.782333 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:57:59.800817 9788 heartbeater.cc:503] Master 127.8.208.126:38755 was elected leader, sending a full tablet report...
I20260929 02:57:59.801108 9790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.788487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38328 (local address 127.8.208.126:38755)
0929 02:57:59.788665 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:59.788669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:59.789158 (+ 489us) server_negotiation.cc:407] Connection header received
0929 02:57:59.789349 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:59.789354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:59.789415 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:59.789576 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:57:59.791070 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.791763 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.793002 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.793478 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.793608 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.796023 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:57:59.796029 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:57:59.796032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:57:59.796061 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:57:59.798441 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:59.799087 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:59.799099 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:59.799102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:59.799224 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:57:59.799584 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:57:59.799592 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:57:59.799597 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:57:59.799900 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:57:59.800152 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:59.800520 (+ 368us) server_negotiation.cc:299] Negotiation successful
0929 02:57:59.800755 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 02:57:59.807219 9598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:57:59.814024 9598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:57:59.838856 10067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.825565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:42545 (local address 127.0.0.1:50044)
0929 02:57:59.826197 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:59.826237 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:59.826408 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:59.831827 (+ 5419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:59.831832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:59.831868 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:59.832229 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.832240 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.834261 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.834269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.836102 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.836113 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.837885 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.837909 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.838081 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.838111 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:57:59.838184 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 02:57:59.838389 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":116,"thread_start_us":293,"threads_started":1}
I20260929 02:57:59.838927 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.826016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:43687 (local address 127.0.0.1:37486)
0929 02:57:59.827011 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:59.827039 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:59.827136 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:59.831020 (+ 3884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:59.831026 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:59.831057 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:59.831629 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.831644 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.833824 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.833829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.835898 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.835909 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.837731 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.837737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.837965 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.838020 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 02:57:59.838145 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 02:57:59.838365 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"mutex_wait_us":1,"thread_start_us":5241,"threads_started":1}
I20260929 02:57:59.838949 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.826598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37486 (local address 127.8.208.66:43687)
0929 02:57:59.829689 (+ 3091us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:59.829701 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:59.829736 (+ 35us) server_negotiation.cc:407] Connection header received
0929 02:57:59.829851 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:59.829859 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:59.830148 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:59.831849 (+ 1701us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:59.832075 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.833681 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.836050 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.837520 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.837673 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.837921 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:59.838437 (+ 516us) server_negotiation.cc:299] Negotiation successful
0929 02:57:59.838618 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2882,"thread_start_us":2753,"threads_started":1}
I20260929 02:57:59.840278 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.825642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.8.208.65:42545)
0929 02:57:59.831176 (+ 5534us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:59.831185 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:59.831269 (+ 84us) server_negotiation.cc:407] Connection header received
0929 02:57:59.831374 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:59.831381 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:59.831619 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:59.832439 (+ 820us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:59.832636 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.834126 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.836244 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.837665 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.839506 (+ 1841us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.839720 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:59.839793 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 02:57:59.839962 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5371,"thread_start_us":5223,"threads_started":1}
I20260929 02:57:59.840883 10069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.826557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:33969 (local address 127.0.0.1:48520)
0929 02:57:59.831603 (+ 5046us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:59.831619 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:59.831743 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:59.832472 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:59.832476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:59.832502 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:59.832845 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.832854 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.834211 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.834217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.837236 (+ 3019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.837247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.840127 (+ 2880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.840133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.840295 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.840326 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:57:59.840415 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 02:57:59.840528 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4895,"mutex_wait_us":83,"thread_start_us":664,"threads_started":1}
I20260929 02:57:59.841555 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.827072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48520 (local address 127.8.208.67:33969)
0929 02:57:59.830736 (+ 3664us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:59.830746 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:59.831869 (+ 1123us) server_negotiation.cc:407] Connection header received
0929 02:57:59.831969 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:59.831976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:59.832281 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:59.832477 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:59.833024 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.834077 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.837380 (+ 3303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.839918 (+ 2538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.840038 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.840248 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:59.841084 (+ 836us) server_negotiation.cc:299] Negotiation successful
0929 02:57:59.841277 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3517,"thread_start_us":3391,"threads_started":1}
I20260929 02:57:59.842923 9857 tablet_service.cc:1467] Processing CreateTablet for tablet abe40a0796744fd08d0f97020830c278 (DEFAULT_TABLE table=test-table [id=7ed95eeb7f3047a9820e7b2ef3a51365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:57:59.844092 9857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abe40a0796744fd08d0f97020830c278. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:59.843644 9723 tablet_service.cc:1467] Processing CreateTablet for tablet abe40a0796744fd08d0f97020830c278 (DEFAULT_TABLE table=test-table [id=7ed95eeb7f3047a9820e7b2ef3a51365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:57:59.844974 9723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abe40a0796744fd08d0f97020830c278. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:59.844901 9990 tablet_service.cc:1467] Processing CreateTablet for tablet abe40a0796744fd08d0f97020830c278 (DEFAULT_TABLE table=test-table [id=7ed95eeb7f3047a9820e7b2ef3a51365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:57:59.846230 9990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abe40a0796744fd08d0f97020830c278. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:57:59.855140 10073 tablet_bootstrap.cc:492] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7: Bootstrap starting.
I20260929 02:57:59.858165 10073 tablet_bootstrap.cc:654] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:59.859109 10074 tablet_bootstrap.cc:492] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da: Bootstrap starting.
I20260929 02:57:59.859815 10073 log.cc:824] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:59.860806 10075 tablet_bootstrap.cc:492] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8: Bootstrap starting.
I20260929 02:57:59.863682 10074 tablet_bootstrap.cc:654] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:59.863725 10075 tablet_bootstrap.cc:654] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8: Neither blocks nor log segments found. Creating new log.
I20260929 02:57:59.864295 10073 tablet_bootstrap.cc:492] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7: No bootstrap required, opened a new log
I20260929 02:57:59.864560 10073 ts_tablet_manager.cc:1397] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260929 02:57:59.864818 10075 log.cc:824] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:59.864866 10074 log.cc:824] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da: Log is configured to *not* fsync() on all Append() calls
I20260929 02:57:59.867856 10075 tablet_bootstrap.cc:492] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8: No bootstrap required, opened a new log
I20260929 02:57:59.868237 10075 ts_tablet_manager.cc:1397] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260929 02:57:59.868507 10074 tablet_bootstrap.cc:492] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da: No bootstrap required, opened a new log
I20260929 02:57:59.868813 10074 ts_tablet_manager.cc:1397] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260929 02:57:59.874402 10073 raft_consensus.cc:348] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db729f9d2ba24ef484077504795746da" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 42545 } } peers { permanent_uuid: "def5352312a04934a14f0bbf0dad01d8" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 33969 } } peers { permanent_uuid: "e6e0a104380a4d07acbf6e949fafcba7" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43687 } }
I20260929 02:57:59.874955 10073 raft_consensus.cc:374] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:59.875082 10073 raft_consensus.cc:729] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6e0a104380a4d07acbf6e949fafcba7, State: Initialized, Role: FOLLOWER
I20260929 02:57:59.875864 10073 consensus_queue.cc:260] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db729f9d2ba24ef484077504795746da" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 42545 } } peers { permanent_uuid: "def5352312a04934a14f0bbf0dad01d8" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 33969 } } peers { permanent_uuid: "e6e0a104380a4d07acbf6e949fafcba7" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43687 } }
I20260929 02:57:59.879092 10075 raft_consensus.cc:348] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db729f9d2ba24ef484077504795746da" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 42545 } } peers { permanent_uuid: "def5352312a04934a14f0bbf0dad01d8" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 33969 } } peers { permanent_uuid: "e6e0a104380a4d07acbf6e949fafcba7" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43687 } }
I20260929 02:57:59.879112 10074 raft_consensus.cc:348] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db729f9d2ba24ef484077504795746da" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 42545 } } peers { permanent_uuid: "def5352312a04934a14f0bbf0dad01d8" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 33969 } } peers { permanent_uuid: "e6e0a104380a4d07acbf6e949fafcba7" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43687 } }
I20260929 02:57:59.879522 10074 raft_consensus.cc:374] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:59.879544 10075 raft_consensus.cc:374] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:57:59.879642 10075 raft_consensus.cc:729] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: def5352312a04934a14f0bbf0dad01d8, State: Initialized, Role: FOLLOWER
I20260929 02:57:59.879657 10074 raft_consensus.cc:729] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db729f9d2ba24ef484077504795746da, State: Initialized, Role: FOLLOWER
I20260929 02:57:59.880371 10074 consensus_queue.cc:260] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db729f9d2ba24ef484077504795746da" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 42545 } } peers { permanent_uuid: "def5352312a04934a14f0bbf0dad01d8" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 33969 } } peers { permanent_uuid: "e6e0a104380a4d07acbf6e949fafcba7" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43687 } }
I20260929 02:57:59.880373 10075 consensus_queue.cc:260] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db729f9d2ba24ef484077504795746da" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 42545 } } peers { permanent_uuid: "def5352312a04934a14f0bbf0dad01d8" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 33969 } } peers { permanent_uuid: "e6e0a104380a4d07acbf6e949fafcba7" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43687 } }
I20260929 02:57:59.886232 9922 heartbeater.cc:503] Master 127.8.208.126:38755 was elected leader, sending a full tablet report...
I20260929 02:57:59.887038 10073 ts_tablet_manager.cc:1428] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7: Time spent starting tablet: real 0.022s user 0.016s sys 0.007s
I20260929 02:57:59.887746 10055 heartbeater.cc:503] Master 127.8.208.126:38755 was elected leader, sending a full tablet report...
I20260929 02:57:59.888921 10075 ts_tablet_manager.cc:1428] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8: Time spent starting tablet: real 0.021s user 0.011s sys 0.007s
I20260929 02:57:59.891151 10074 ts_tablet_manager.cc:1428] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da: Time spent starting tablet: real 0.022s user 0.019s sys 0.003s
I20260929 02:57:59.985616 10079 raft_consensus.cc:482] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:57:59.985975 10079 raft_consensus.cc:504] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db729f9d2ba24ef484077504795746da" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 42545 } } peers { permanent_uuid: "def5352312a04934a14f0bbf0dad01d8" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 33969 } } peers { permanent_uuid: "e6e0a104380a4d07acbf6e949fafcba7" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43687 } }
I20260929 02:57:59.987560 10079 leader_election.cc:288] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db729f9d2ba24ef484077504795746da (127.8.208.65:42545), def5352312a04934a14f0bbf0dad01d8 (127.8.208.67:33969)
I20260929 02:57:59.994545 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.988003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:45571 (local address 127.8.208.65:42545)
0929 02:57:59.988178 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:59.988182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:59.988726 (+ 544us) server_negotiation.cc:407] Connection header received
0929 02:57:59.989076 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:59.989081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:59.989161 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:59.989266 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:59.990067 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.990877 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.992747 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.993607 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.993794 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.993874 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:59.994175 (+ 301us) server_negotiation.cc:299] Negotiation successful
0929 02:57:59.994298 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:57:59.994612 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.988956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:45563 (local address 127.8.208.67:33969)
0929 02:57:59.989162 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 02:57:59.989191 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0929 02:57:59.989416 (+ 225us) server_negotiation.cc:407] Connection header received
0929 02:57:59.989624 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:57:59.989632 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:57:59.989692 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:57:59.989795 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:57:59.990452 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.991228 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.992916 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.993710 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.993884 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.993958 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 02:57:59.994245 (+ 287us) server_negotiation.cc:299] Negotiation successful
0929 02:57:59.994369 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 02:57:59.994756 10082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.987838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:42545 (local address 127.8.208.66:45571)
0929 02:57:59.988619 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:59.988644 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:59.988854 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:59.989435 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:59.989438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:59.989510 (+ 72us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:59.989852 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.989861 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.991013 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.991021 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.992591 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.992603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.993823 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.993829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.993992 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.994018 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 02:57:59.994144 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 02:57:59.994272 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":63,"spinlock_wait_cycles":18176,"thread_start_us":406,"threads_started":1}
I20260929 02:57:59.994866 10083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:57:59.988703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:33969 (local address 127.8.208.66:45563)
0929 02:57:59.989310 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0929 02:57:59.989323 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:57:59.989478 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:57:59.989816 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:57:59.989820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:57:59.989859 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:57:59.990306 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.990322 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.991361 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.991368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.992766 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:57:59.992777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:57:59.993923 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:57:59.993929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:57:59.994100 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:57:59.994121 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 02:57:59.994221 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 02:57:59.994338 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":1,"thread_start_us":292,"threads_started":1}
I20260929 02:57:59.995774 10010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe40a0796744fd08d0f97020830c278" candidate_uuid: "e6e0a104380a4d07acbf6e949fafcba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "def5352312a04934a14f0bbf0dad01d8" is_pre_election: true
I20260929 02:57:59.995771 9743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe40a0796744fd08d0f97020830c278" candidate_uuid: "e6e0a104380a4d07acbf6e949fafcba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db729f9d2ba24ef484077504795746da" is_pre_election: true
I20260929 02:57:59.996251 10010 raft_consensus.cc:2454] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6e0a104380a4d07acbf6e949fafcba7 in term 0.
I20260929 02:57:59.996249 9743 raft_consensus.cc:2454] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6e0a104380a4d07acbf6e949fafcba7 in term 0.
I20260929 02:57:59.996865 9810 leader_election.cc:302] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: def5352312a04934a14f0bbf0dad01d8, e6e0a104380a4d07acbf6e949fafcba7; no voters:
I20260929 02:57:59.997255 10079 raft_consensus.cc:2781] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:57:59.997411 10079 raft_consensus.cc:482] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:57:59.997534 10079 raft_consensus.cc:3037] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:00.000047 10079 raft_consensus.cc:504] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db729f9d2ba24ef484077504795746da" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 42545 } } peers { permanent_uuid: "def5352312a04934a14f0bbf0dad01d8" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 33969 } } peers { permanent_uuid: "e6e0a104380a4d07acbf6e949fafcba7" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43687 } }
I20260929 02:58:00.000633 10079 leader_election.cc:288] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [CANDIDATE]: Term 1 election: Requested vote from peers db729f9d2ba24ef484077504795746da (127.8.208.65:42545), def5352312a04934a14f0bbf0dad01d8 (127.8.208.67:33969)
I20260929 02:58:00.001421 10010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe40a0796744fd08d0f97020830c278" candidate_uuid: "e6e0a104380a4d07acbf6e949fafcba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "def5352312a04934a14f0bbf0dad01d8"
I20260929 02:58:00.001392 9743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abe40a0796744fd08d0f97020830c278" candidate_uuid: "e6e0a104380a4d07acbf6e949fafcba7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db729f9d2ba24ef484077504795746da"
I20260929 02:58:00.001634 10010 raft_consensus.cc:3037] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:00.001619 9743 raft_consensus.cc:3037] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da [term 0 FOLLOWER]: Advancing to term 1
W20260929 02:58:00.003253 10056 tablet.cc:2369] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:58:00.003960 10010 raft_consensus.cc:2454] T abe40a0796744fd08d0f97020830c278 P def5352312a04934a14f0bbf0dad01d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6e0a104380a4d07acbf6e949fafcba7 in term 1.
I20260929 02:58:00.004293 9743 raft_consensus.cc:2454] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6e0a104380a4d07acbf6e949fafcba7 in term 1.
I20260929 02:58:00.004417 9810 leader_election.cc:302] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: def5352312a04934a14f0bbf0dad01d8, e6e0a104380a4d07acbf6e949fafcba7; no voters:
I20260929 02:58:00.004760 10079 raft_consensus.cc:2781] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:00.005268 10079 raft_consensus.cc:686] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [term 1 LEADER]: Becoming Leader. State: Replica: e6e0a104380a4d07acbf6e949fafcba7, State: Running, Role: LEADER
I20260929 02:58:00.005730 10079 consensus_queue.cc:237] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db729f9d2ba24ef484077504795746da" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 42545 } } peers { permanent_uuid: "def5352312a04934a14f0bbf0dad01d8" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 33969 } } peers { permanent_uuid: "e6e0a104380a4d07acbf6e949fafcba7" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43687 } }
I20260929 02:58:00.010573 9597 catalog_manager.cc:5495] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6e0a104380a4d07acbf6e949fafcba7 (127.8.208.66). New cstate: current_term: 1 leader_uuid: "e6e0a104380a4d07acbf6e949fafcba7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db729f9d2ba24ef484077504795746da" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 42545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "def5352312a04934a14f0bbf0dad01d8" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 33969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6e0a104380a4d07acbf6e949fafcba7" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43687 } health_report { overall_health: HEALTHY } } }
W20260929 02:58:00.021459 9789 tablet.cc:2369] T abe40a0796744fd08d0f97020830c278 P db729f9d2ba24ef484077504795746da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:58:00.024307 9923 tablet.cc:2369] T abe40a0796744fd08d0f97020830c278 P e6e0a104380a4d07acbf6e949fafcba7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 02:58:00 dist-test-slave-m1rv krb5kdc[9554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:58:00.052421 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:00.038020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50054 (local address 127.8.208.65:42545)
0929 02:58:00.038180 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:00.038184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:00.038656 (+ 472us) server_negotiation.cc:407] Connection header received
0929 02:58:00.038869 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:00.038873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:00.038944 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:00.039050 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:00.040546 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:00.041380 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:00.042591 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:00.043027 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:00.043161 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:00.047021 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:00.047034 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:00.047049 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:00.047087 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:00.049816 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:00.050452 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:00.050459 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:00.050461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:00.050577 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:00.050932 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:00.050938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:00.050941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:00.051284 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:00.051593 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:00.051869 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 02:58:00.052076 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Sep 29 02:58:00 dist-test-slave-m1rv krb5kdc[9554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650680, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:58:00 dist-test-slave-m1rv krb5kdc[9554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650680, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:58:00.095903 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:00.082467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50064 (local address 127.8.208.65:42545)
0929 02:58:00.082665 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:00.082672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:00.083034 (+ 362us) server_negotiation.cc:407] Connection header received
0929 02:58:00.083266 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:00.083270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:00.083333 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:00.083463 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:00.084876 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:00.085757 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:00.086997 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:00.087413 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:00.087547 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:00.091338 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:00.091344 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:00.091347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:00.091381 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:00.093815 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:00.094349 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:00.094353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:00.094355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:00.094482 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:00.094831 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:00.094834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:00.094836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:00.095074 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:00.095268 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:00.095515 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 02:58:00.095670 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Sep 29 02:58:00 dist-test-slave-m1rv krb5kdc[9554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:58:00 dist-test-slave-m1rv krb5kdc[9554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:58:00.137897 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:00.125090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50070 (local address 127.8.208.65:42545)
0929 02:58:00.125328 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:00.125334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:00.125500 (+ 166us) server_negotiation.cc:407] Connection header received
0929 02:58:00.125644 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:00.125648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:00.125716 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:00.125800 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:00.127067 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:00.127740 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:00.128860 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:00.129347 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:00.129446 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:00.133390 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:00.133395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:00.133398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:00.133434 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:00.135692 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:00.136251 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:00.136254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:00.136257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:00.136371 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:00.136671 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:00.136674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:00.136676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:00.136936 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:00.137112 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:00.137391 (+ 279us) server_negotiation.cc:299] Negotiation successful
0929 02:58:00.137550 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
W20260929 02:58:00.139662 9703 tablet_service.cc:2908] Rejecting scan request for tablet abe40a0796744fd08d0f97020830c278: Uninitialized: clean time has not yet been initialized
I20260929 02:58:00.142488 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 9660
I20260929 02:58:00.154117 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:00.143319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50080 (local address 127.8.208.65:42545)
0929 02:58:00.143522 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:00.143530 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:00.143781 (+ 251us) server_negotiation.cc:407] Connection header received
0929 02:58:00.143889 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:00.143892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:00.143955 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:00.144030 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:00.145117 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:00.145826 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:00.146940 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:00.147306 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:00.147411 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:00.149591 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:00.149595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:00.149598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:00.149624 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:00.151950 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:00.152522 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:00.152526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:00.152528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:00.152634 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:00.152928 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:00.152930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:00.152932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:00.153233 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:00.153452 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:00.153644 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 02:58:00.153794 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:58:00.154744 9783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:00.254765 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 9660
I20260929 02:58:00.272226 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 9794
Sep 29 02:58:00 dist-test-slave-m1rv krb5kdc[9554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 02:58:00.288571 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:00.273218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37492 (local address 127.8.208.66:43687)
0929 02:58:00.273422 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:00.273428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:00.273448 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:58:00.273506 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:00.273512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:00.273589 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:00.273689 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:00.274930 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:00.275989 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:00.277253 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:00.277940 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:00.278054 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:00.281973 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:00.281983 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:00.281999 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:00.282037 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:00.285843 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:00.286517 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:00.286527 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:00.286530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:00.286672 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:00.287038 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:00.287044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:00.287047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:00.287402 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:00.287703 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:00.287989 (+ 286us) server_negotiation.cc:299] Negotiation successful
0929 02:58:00.288242 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:58:00.289361 9917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:00.379495 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 9794
I20260929 02:58:00.396118 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 9927
Sep 29 02:58:00 dist-test-slave-m1rv krb5kdc[9554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 02:58:00.410856 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:00.396860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48524 (local address 127.8.208.67:33969)
0929 02:58:00.397024 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:00.397028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:00.397046 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:58:00.397252 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:00.397256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:00.397316 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:00.397403 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:00.398537 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:00.399233 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:00.400531 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:00.400906 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:00.400997 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:00.405439 (+ 4442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:00.405456 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:00.405476 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:00.405516 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:00.408418 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:00.408988 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:00.408994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:00.408997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:00.409112 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:00.409503 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:00.409506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:00.409508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:00.409861 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:00.410070 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:00.410326 (+ 256us) server_negotiation.cc:299] Negotiation successful
0929 02:58:00.410490 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:58:00.411497 10050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:00.500146 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 9927
I20260929 02:58:00.516124 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 9567
I20260929 02:58:00.517299 9629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:00.625324 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 9567
2026-09-29T02:58:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4116 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:58:00 dist-test-slave-m1rv krb5kdc[10103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:58:00 dist-test-slave-m1rv krb5kdc[10103](info): set up 2 sockets
Sep 29 02:58:00 dist-test-slave-m1rv krb5kdc[10103](info): commencing operation
krb5kdc: starting...
W20260929 02:58:01.721201 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.003s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:58:01 dist-test-slave-m1rv krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:58:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:58:01.900707 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:42781
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:42781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:02.209239 10116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:02.209607 10116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:02.209777 10116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:02.216953 10116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:02.217109 10116 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 02:58:02.217223 10116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:02.217345 10116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:02.217439 10116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:02.217526 10116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:02.227149 10116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:42781
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:42781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:02.228381 10116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:02.229707 10116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:02.241529 10122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:02.241566 10124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:02.241590 10121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:02.242308 10116 server_base.cc:1034] running on GCE node
I20260929 02:58:02.242957 10116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:02.244211 10116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:02.245420 10116 hybrid_clock.cc:648] HybridClock initialized: now 1790650682245349 us; error 29 us; skew 500 ppm
I20260929 02:58:02.245755 10116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:02 dist-test-slave-m1rv krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650682, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:02.248781 10116 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:58:02.250180 10116 webserver.cc:460] Webserver started at http://127.8.208.126:46227/ using document root <none> and password file <none>
I20260929 02:58:02.250834 10116 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:02.250918 10116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:02.251190 10116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:02.254091 10116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "df77afafbd25487f917596aa0d08c076"
format_stamp: "Formatted at 2026-09-29 02:58:02 on dist-test-slave-m1rv"
server_key: "f94f2ecb3cb71a3df534282874226950"
server_key_iv: "529ba2bbe843ab1fa540237a32b4f6de"
server_key_version: "encryptionkey@0"
I20260929 02:58:02.254806 10116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "df77afafbd25487f917596aa0d08c076"
format_stamp: "Formatted at 2026-09-29 02:58:02 on dist-test-slave-m1rv"
server_key: "f94f2ecb3cb71a3df534282874226950"
server_key_iv: "529ba2bbe843ab1fa540237a32b4f6de"
server_key_version: "encryptionkey@0"
I20260929 02:58:02.259812 10116 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260929 02:58:02.263151 10131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:02.264415 10116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:02.264591 10116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "df77afafbd25487f917596aa0d08c076"
format_stamp: "Formatted at 2026-09-29 02:58:02 on dist-test-slave-m1rv"
server_key: "f94f2ecb3cb71a3df534282874226950"
server_key_iv: "529ba2bbe843ab1fa540237a32b4f6de"
server_key_version: "encryptionkey@0"
I20260929 02:58:02.264801 10116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:02.286418 10116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:02.290450 10116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:02.290726 10116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:02.313331 10116 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:42781
I20260929 02:58:02.313434 10192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:42781 every 8 connection(s)
I20260929 02:58:02.314762 10116 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:02.316751 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 10116
I20260929 02:58:02.316989 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:02.317888 9025 external_mini_cluster.cc:1428] Setting key d36504e1169d3017df1e02025e08437a
I20260929 02:58:02.319099 10193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:58:02 dist-test-slave-m1rv krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:02.332675 10193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076: Bootstrap starting.
I20260929 02:58:02.336700 10193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:02.338006 10193 log.cc:824] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:02.341285 10193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076: No bootstrap required, opened a new log
I20260929 02:58:02.341688 10196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:02.319810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47530 (local address 127.8.208.126:42781)
0929 02:58:02.320622 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:02.320637 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:02.320686 (+ 49us) server_negotiation.cc:407] Connection header received
0929 02:58:02.322625 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:02.322652 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:02.323173 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:02.323621 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:02.324879 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:02.326966 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:02.328279 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:02.329350 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:02.329557 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:02.333482 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:02.333492 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:02.333510 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:02.333553 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:02.337384 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:02.338066 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:02.338073 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:02.338081 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:02.338262 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:02.338675 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:02.338683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:02.338687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:02.339240 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:02.339580 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:02.339916 (+ 336us) server_negotiation.cc:299] Negotiation successful
0929 02:58:02.340306 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":340,"threads_started":1}
I20260929 02:58:02.349161 10193 raft_consensus.cc:348] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df77afafbd25487f917596aa0d08c076" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42781 } }
I20260929 02:58:02.349529 10193 raft_consensus.cc:374] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:02.349604 10193 raft_consensus.cc:729] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df77afafbd25487f917596aa0d08c076, State: Initialized, Role: FOLLOWER
I20260929 02:58:02.350071 10193 consensus_queue.cc:260] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df77afafbd25487f917596aa0d08c076" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42781 } }
I20260929 02:58:02.350358 10193 raft_consensus.cc:388] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:02.350504 10193 raft_consensus.cc:482] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:02.350692 10193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:02.352703 10193 raft_consensus.cc:504] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df77afafbd25487f917596aa0d08c076" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42781 } }
I20260929 02:58:02.353121 10193 leader_election.cc:302] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df77afafbd25487f917596aa0d08c076; no voters:
I20260929 02:58:02.353734 10193 leader_election.cc:288] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:02.353880 10198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:02.354619 10198 raft_consensus.cc:686] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [term 1 LEADER]: Becoming Leader. State: Replica: df77afafbd25487f917596aa0d08c076, State: Running, Role: LEADER
I20260929 02:58:02.355293 10198 consensus_queue.cc:237] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df77afafbd25487f917596aa0d08c076" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42781 } }
I20260929 02:58:02.355949 10193 sys_catalog.cc:564] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:02.358399 10200 sys_catalog.cc:455] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df77afafbd25487f917596aa0d08c076. Latest consensus state: current_term: 1 leader_uuid: "df77afafbd25487f917596aa0d08c076" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df77afafbd25487f917596aa0d08c076" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42781 } } }
I20260929 02:58:02.358819 10200 sys_catalog.cc:458] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:02.358579 10199 sys_catalog.cc:455] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df77afafbd25487f917596aa0d08c076" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df77afafbd25487f917596aa0d08c076" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42781 } } }
I20260929 02:58:02.358945 10199 sys_catalog.cc:458] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:02.361779 10203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:02.367210 10203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:02.375712 10203 catalog_manager.cc:1347] Generated new cluster ID: 325895c6c9c548eb8f427801c1d90c55
I20260929 02:58:02.375823 10203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:02.399986 10203 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:02.401939 10203 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:02.417794 10203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076: Generated new TSK 0
I20260929 02:58:02.418869 10203 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 02:58:02.473059 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:42781
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:02.788225 10219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:02.788532 10219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:02.788695 10219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:02.796022 10219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:02.796167 10219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:02.796314 10219 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:58:02.796465 10219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:58:02.806007 10219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.208.126:42781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:02.807220 10219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:02.808504 10219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:02.821410 10225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:02.821511 10224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:02.821488 10227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:02.822136 10219 server_base.cc:1034] running on GCE node
I20260929 02:58:02.822728 10219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:02.823558 10219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:02.824801 10219 hybrid_clock.cc:648] HybridClock initialized: now 1790650682824744 us; error 57 us; skew 500 ppm
I20260929 02:58:02.825095 10219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:02 dist-test-slave-m1rv krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650682, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:02.828114 10219 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 02:58:02.829509 10219 webserver.cc:460] Webserver started at http://127.8.208.65:45971/ using document root <none> and password file <none>
I20260929 02:58:02.830178 10219 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:02.830309 10219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:02.830706 10219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:02.833550 10219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "f6b589b5c9994f418c9ea1c570c0c40a"
format_stamp: "Formatted at 2026-09-29 02:58:02 on dist-test-slave-m1rv"
server_key: "dbafeaa26cb5b462aa0c61cab6289f09"
server_key_iv: "62e408e7c4edaceb2601c41555238777"
server_key_version: "encryptionkey@0"
I20260929 02:58:02.834316 10219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "f6b589b5c9994f418c9ea1c570c0c40a"
format_stamp: "Formatted at 2026-09-29 02:58:02 on dist-test-slave-m1rv"
server_key: "dbafeaa26cb5b462aa0c61cab6289f09"
server_key_iv: "62e408e7c4edaceb2601c41555238777"
server_key_version: "encryptionkey@0"
I20260929 02:58:02.839308 10219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:58:02.842890 10234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:02.844137 10219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:58:02.844319 10219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "f6b589b5c9994f418c9ea1c570c0c40a"
format_stamp: "Formatted at 2026-09-29 02:58:02 on dist-test-slave-m1rv"
server_key: "dbafeaa26cb5b462aa0c61cab6289f09"
server_key_iv: "62e408e7c4edaceb2601c41555238777"
server_key_version: "encryptionkey@0"
I20260929 02:58:02.844543 10219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:02.873309 10219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:02.877262 10219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:02.877537 10219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:02.880071 10219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:02.880182 10219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:02.880354 10219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:02.880435 10219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:58:02 dist-test-slave-m1rv krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650682, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:02.902871 10243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:02.880754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:37899 (local address 127.8.208.126:42781)
0929 02:58:02.883079 (+ 2325us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:02.883086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:02.883109 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:58:02.884615 (+ 1506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:02.884620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:02.884734 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:02.884858 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:02.887122 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:02.888185 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:02.890786 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:02.891337 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:02.891481 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:02.895981 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:02.895986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:02.895989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:02.896020 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:02.899408 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:02.900326 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:02.900331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:02.900335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:02.900456 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:02.900879 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:02.900884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:02.900887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:02.901154 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:02.901364 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:02.902301 (+ 937us) server_negotiation.cc:299] Negotiation successful
0929 02:58:02.902516 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2198,"thread_start_us":2089,"threads_started":1}
I20260929 02:58:02.904312 10244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:02.881418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:42781 (local address 127.8.208.65:37899)
0929 02:58:02.882319 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:02.882368 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:02.884299 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:02.885091 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:02.885098 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:02.885886 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:02.886892 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:02.886927 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:02.888326 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:02.888331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:02.890631 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:02.890643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:02.891592 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:02.891599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:02.891958 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:02.892700 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:02.892721 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:02.895700 (+ 2979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:02.899537 (+ 3837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:02.899546 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:02.899566 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:02.900198 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:02.900568 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:02.900573 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:02.900576 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:02.900762 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:02.901353 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:02.901360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:02.901866 (+ 506us) client_negotiation.cc:769] Sending connection context
0929 02:58:02.902221 (+ 355us) client_negotiation.cc:240] Negotiation successful
0929 02:58:02.902721 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":443,"threads_started":1}
I20260929 02:58:02.927802 10243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:02.909988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:56215 (local address 127.8.208.126:42781)
0929 02:58:02.910219 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:02.910228 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:02.913961 (+ 3733us) server_negotiation.cc:407] Connection header received
0929 02:58:02.914166 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:02.914170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:02.914229 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:02.914340 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:02.915809 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:02.916736 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:02.918798 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:02.919343 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:02.919472 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:02.921969 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:02.921973 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:02.921977 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:02.922005 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:02.924830 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:02.925784 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:02.925790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:02.925793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:02.925932 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:02.926309 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:02.926316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:02.926320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:02.926672 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:02.926825 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:02.927219 (+ 394us) server_negotiation.cc:299] Negotiation successful
0929 02:58:02.927435 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:58:02.927911 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:02.909896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:42781 (local address 127.8.208.65:56215)
0929 02:58:02.913822 (+ 3926us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:02.913848 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:02.914043 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:02.914364 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:02.914374 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:02.915052 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:02.915604 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:02.915621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:02.916933 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:02.916945 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:02.918657 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:02.918669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:02.919795 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:02.919807 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:02.920029 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:02.920791 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:02.920807 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:02.921742 (+ 935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:02.924973 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:02.924978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:02.924982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:02.925624 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:02.926056 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:02.926060 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:02.926064 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:02.926191 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:02.926790 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:02.926795 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:02.926983 (+ 188us) client_negotiation.cc:769] Sending connection context
0929 02:58:02.927122 (+ 139us) client_negotiation.cc:240] Negotiation successful
0929 02:58:02.927443 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3720,"thread_start_us":3570,"threads_started":1}
I20260929 02:58:02.937448 10219 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:44127
I20260929 02:58:02.937469 10354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:44127 every 8 connection(s)
I20260929 02:58:02.938624 10219 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:58:02.940685 10355 heartbeater.cc:347] Connected to a master server at 127.8.208.126:42781
I20260929 02:58:02.940896 10355 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:02.941608 10355 heartbeater.cc:511] Master 127.8.208.126:42781 requested a full tablet report, sending...
I20260929 02:58:02.942983 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 10219
I20260929 02:58:02.943198 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:58:02.943558 10148 ts_manager.cc:194] Registered new tserver with Master: f6b589b5c9994f418c9ea1c570c0c40a (127.8.208.65:44127)
I20260929 02:58:02.943847 9025 external_mini_cluster.cc:1428] Setting key f185c088469f9e4880264be09c02b523
I20260929 02:58:02.945390 10148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:37899
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 02:58:02.982378 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:42781
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:03.291028 10359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:03.291304 10359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:03.291400 10359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:03.298647 10359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:03.298761 10359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:03.298964 10359 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:58:03.299082 10359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:58:03.308677 10359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.208.126:42781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:03.309975 10359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:03.311268 10359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:03.325563 10367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:03.325563 10365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:03.326303 10364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:03.326578 10359 server_base.cc:1034] running on GCE node
I20260929 02:58:03.327289 10359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:03.328333 10359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:03.329568 10359 hybrid_clock.cc:648] HybridClock initialized: now 1790650683329489 us; error 51 us; skew 500 ppm
I20260929 02:58:03.329888 10359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:03 dist-test-slave-m1rv krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650683, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:03.333561 10359 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 02:58:03.334882 10359 webserver.cc:460] Webserver started at http://127.8.208.66:44349/ using document root <none> and password file <none>
I20260929 02:58:03.335548 10359 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:03.335657 10359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:03.335979 10359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:03.338769 10359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "e8c963293ec54ac28537717e51010e48"
format_stamp: "Formatted at 2026-09-29 02:58:03 on dist-test-slave-m1rv"
server_key: "558070bdb6be01fb17e755a2328c2b2b"
server_key_iv: "ce851b8c8bf6aa399852feb54ecb605c"
server_key_version: "encryptionkey@0"
I20260929 02:58:03.339524 10359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "e8c963293ec54ac28537717e51010e48"
format_stamp: "Formatted at 2026-09-29 02:58:03 on dist-test-slave-m1rv"
server_key: "558070bdb6be01fb17e755a2328c2b2b"
server_key_iv: "ce851b8c8bf6aa399852feb54ecb605c"
server_key_version: "encryptionkey@0"
I20260929 02:58:03.344534 10359 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 02:58:03.347890 10374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:03.349347 10359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:58:03.349468 10359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "e8c963293ec54ac28537717e51010e48"
format_stamp: "Formatted at 2026-09-29 02:58:03 on dist-test-slave-m1rv"
server_key: "558070bdb6be01fb17e755a2328c2b2b"
server_key_iv: "ce851b8c8bf6aa399852feb54ecb605c"
server_key_version: "encryptionkey@0"
I20260929 02:58:03.349634 10359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:03.379750 10359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:03.383653 10359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:03.383872 10359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:03.386499 10359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:03.386598 10359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:03.386757 10359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:03.386819 10359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:58:03 dist-test-slave-m1rv krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650683, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:03.410262 10243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:03.386925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:60911 (local address 127.8.208.126:42781)
0929 02:58:03.387153 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:03.387161 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:03.388505 (+ 1344us) server_negotiation.cc:407] Connection header received
0929 02:58:03.390702 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:03.390710 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:03.390824 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:03.390969 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:03.393502 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.394622 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.397318 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.397811 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.397938 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:03.402563 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:03.402572 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:03.402579 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:03.402631 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:03.406570 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:03.407408 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:03.407417 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:03.407423 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:03.407603 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:03.408105 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:03.408112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:03.408117 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:03.408587 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:03.408799 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:03.409679 (+ 880us) server_negotiation.cc:299] Negotiation successful
0929 02:58:03.409916 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 02:58:03.411284 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:03.387596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:42781 (local address 127.8.208.66:60911)
0929 02:58:03.388339 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:03.388379 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:03.390308 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:03.391266 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:03.391276 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:03.392088 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:03.393307 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:03.393340 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.394787 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.394799 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:03.397111 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:03.397126 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.398130 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.398141 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:03.398500 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:03.399350 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:03.399387 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:03.402317 (+ 2930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:03.406722 (+ 4405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:03.406728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:03.406741 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:03.407277 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:03.407727 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:03.407730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:03.407732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:03.407973 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:03.408763 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:03.408769 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:03.409105 (+ 336us) client_negotiation.cc:769] Sending connection context
0929 02:58:03.409577 (+ 472us) client_negotiation.cc:240] Negotiation successful
0929 02:58:03.410121 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":355,"threads_started":1}
I20260929 02:58:03.434563 10243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:03.418644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:47909 (local address 127.8.208.126:42781)
0929 02:58:03.418904 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:03.418912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:03.419447 (+ 535us) server_negotiation.cc:407] Connection header received
0929 02:58:03.419584 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:03.419592 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:03.419691 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:03.419820 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:03.420726 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.422070 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.424625 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.425146 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.425302 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:03.428120 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:03.428127 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:03.428132 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:03.428169 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:03.431496 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:03.432413 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:03.432419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:03.432423 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:03.432579 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:03.432951 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:03.432958 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:03.432961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:03.433357 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:03.433515 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:03.433972 (+ 457us) server_negotiation.cc:299] Negotiation successful
0929 02:58:03.434204 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:58:03.434844 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:03.418645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:42781 (local address 127.8.208.66:47909)
0929 02:58:03.419328 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:03.419359 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:03.419479 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:03.419855 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:03.419858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:03.420273 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:03.420582 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:03.420591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.422281 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.422291 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:03.424449 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:03.424472 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.425401 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.425414 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:03.425667 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:03.426542 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:03.426577 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:03.427846 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:03.431614 (+ 3768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:03.431621 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:03.431624 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:03.432274 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:03.432699 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:03.432704 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:03.432708 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:03.432837 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:03.433491 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:03.433496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:03.433696 (+ 200us) client_negotiation.cc:769] Sending connection context
0929 02:58:03.433882 (+ 186us) client_negotiation.cc:240] Negotiation successful
0929 02:58:03.434338 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":384,"threads_started":1}
I20260929 02:58:03.442198 10359 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:38137
I20260929 02:58:03.442224 10493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:38137 every 8 connection(s)
I20260929 02:58:03.443421 10359 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:58:03.445533 10494 heartbeater.cc:347] Connected to a master server at 127.8.208.126:42781
I20260929 02:58:03.445716 10494 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:03.446293 10494 heartbeater.cc:511] Master 127.8.208.126:42781 requested a full tablet report, sending...
I20260929 02:58:03.447645 10148 ts_manager.cc:194] Registered new tserver with Master: e8c963293ec54ac28537717e51010e48 (127.8.208.66:38137)
I20260929 02:58:03.448436 10148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:60911
I20260929 02:58:03.449013 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 10359
I20260929 02:58:03.449230 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:58:03.449709 9025 external_mini_cluster.cc:1428] Setting key 7faa5a979c942bd13dcd7f8818a60101
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 02:58:03.489053 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:42781
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:03.798394 10498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:03.798668 10498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:03.798839 10498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:03.806123 10498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:03.806241 10498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:03.806391 10498 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:58:03.806531 10498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:58:03.816237 10498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.208.126:42781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:03.817498 10498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:03.818782 10498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:03.831948 10503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:03.831974 10504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:03.831984 10506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:03.832787 10498 server_base.cc:1034] running on GCE node
I20260929 02:58:03.833412 10498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:03.834251 10498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:03.835458 10498 hybrid_clock.cc:648] HybridClock initialized: now 1790650683835408 us; error 30 us; skew 500 ppm
I20260929 02:58:03.835698 10498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:03 dist-test-slave-m1rv krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650683, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:03.838666 10498 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 02:58:03.840039 10498 webserver.cc:460] Webserver started at http://127.8.208.67:40969/ using document root <none> and password file <none>
I20260929 02:58:03.840698 10498 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:03.840816 10498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:03.841238 10498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:03.843976 10498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "5e62477c4f9140649ffd57d0db5351a3"
format_stamp: "Formatted at 2026-09-29 02:58:03 on dist-test-slave-m1rv"
server_key: "dd6b889c95780aac47efcf778cce3bf9"
server_key_iv: "127523f08c0d965c296e810794b95507"
server_key_version: "encryptionkey@0"
I20260929 02:58:03.844738 10498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "5e62477c4f9140649ffd57d0db5351a3"
format_stamp: "Formatted at 2026-09-29 02:58:03 on dist-test-slave-m1rv"
server_key: "dd6b889c95780aac47efcf778cce3bf9"
server_key_iv: "127523f08c0d965c296e810794b95507"
server_key_version: "encryptionkey@0"
I20260929 02:58:03.849773 10498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260929 02:58:03.853235 10513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:03.854584 10498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:58:03.854746 10498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "5e62477c4f9140649ffd57d0db5351a3"
format_stamp: "Formatted at 2026-09-29 02:58:03 on dist-test-slave-m1rv"
server_key: "dd6b889c95780aac47efcf778cce3bf9"
server_key_iv: "127523f08c0d965c296e810794b95507"
server_key_version: "encryptionkey@0"
I20260929 02:58:03.854964 10498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:03.872409 10498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:03.876957 10498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:03.877280 10498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:03.881487 10498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:03.881631 10498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:03.881778 10498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:03.881841 10498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:58:03 dist-test-slave-m1rv krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650683, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:03.904042 10243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:03.881740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:42623 (local address 127.8.208.126:42781)
0929 02:58:03.881930 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:03.881936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:03.883435 (+ 1499us) server_negotiation.cc:407] Connection header received
0929 02:58:03.885768 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:03.885775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:03.885865 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:03.885994 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:03.887461 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.888220 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.890351 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.890950 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.891063 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:03.896285 (+ 5222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:03.896292 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:03.896296 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:03.896338 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:03.899966 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:03.900748 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:03.900751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:03.900754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:03.900874 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:03.901401 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:03.901408 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:03.901413 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:03.901814 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:03.902019 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:03.903117 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0929 02:58:03.903513 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:58:03.904687 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:03.882337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:42781 (local address 127.8.208.67:42623)
0929 02:58:03.883200 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:03.883259 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:03.885489 (+ 2230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:03.886233 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:03.886242 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:03.886729 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:03.887285 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:03.887301 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.888382 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.888399 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:03.890109 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:03.890133 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.891476 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.891485 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:03.891811 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:03.892692 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:03.892714 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:03.896062 (+ 3348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:03.900127 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:03.900135 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:03.900152 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:03.900595 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:03.901000 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:03.901005 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:03.901009 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:03.901283 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:03.901990 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:03.901997 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:03.902535 (+ 538us) client_negotiation.cc:769] Sending connection context
0929 02:58:03.902912 (+ 377us) client_negotiation.cc:240] Negotiation successful
0929 02:58:03.903382 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":361,"threads_started":1}
I20260929 02:58:03.925987 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:03.910792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:42781 (local address 127.8.208.67:40397)
0929 02:58:03.911400 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:03.911415 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:03.911587 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:03.912262 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:03.912267 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:03.912716 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:03.912992 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:03.913001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.914190 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.914197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:03.915249 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:03.915260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.916383 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.916414 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:03.916668 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:03.917451 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:03.917494 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:03.918954 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:03.922658 (+ 3704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:03.922663 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:03.922667 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:03.923300 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:03.923724 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:03.923732 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:03.923736 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:03.923935 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:03.924553 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:03.924558 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:03.924828 (+ 270us) client_negotiation.cc:769] Sending connection context
0929 02:58:03.924998 (+ 170us) client_negotiation.cc:240] Negotiation successful
0929 02:58:03.925495 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":369,"threads_started":1}
I20260929 02:58:03.927066 10243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:03.911116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:40397 (local address 127.8.208.126:42781)
0929 02:58:03.911299 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:03.911305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:03.911524 (+ 219us) server_negotiation.cc:407] Connection header received
0929 02:58:03.912056 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:03.912061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:03.912136 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:03.912240 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:03.913129 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.914062 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.915385 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.916133 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.916275 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:03.919173 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:03.919179 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:03.919185 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:03.919230 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:03.922528 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:03.923410 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:03.923416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:03.923419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:03.923580 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:03.924085 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:03.924090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:03.924094 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:03.924408 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:03.924580 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:03.926340 (+ 1760us) server_negotiation.cc:299] Negotiation successful
0929 02:58:03.926593 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:58:03.934547 10498 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:35673
I20260929 02:58:03.934571 10632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:35673 every 8 connection(s)
I20260929 02:58:03.935739 10498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:58:03.936120 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 10498
I20260929 02:58:03.936318 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:58:03.936924 9025 external_mini_cluster.cc:1428] Setting key f741a2b6bf5220866dc5e55da6e411d3
I20260929 02:58:03.938439 10633 heartbeater.cc:347] Connected to a master server at 127.8.208.126:42781
I20260929 02:58:03.938637 10633 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:03.939364 10633 heartbeater.cc:511] Master 127.8.208.126:42781 requested a full tablet report, sending...
I20260929 02:58:03.940830 10147 ts_manager.cc:194] Registered new tserver with Master: 5e62477c4f9140649ffd57d0db5351a3 (127.8.208.67:35673)
I20260929 02:58:03.941972 10147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:42623
I20260929 02:58:03.948037 10355 heartbeater.cc:503] Master 127.8.208.126:42781 was elected leader, sending a full tablet report...
I20260929 02:58:03.951759 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:58:03 dist-test-slave-m1rv krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650683, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:58:03 dist-test-slave-m1rv krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650683, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:03.997462 10243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:03.984098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47532 (local address 127.8.208.126:42781)
0929 02:58:03.984328 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:03.984335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:03.984607 (+ 272us) server_negotiation.cc:407] Connection header received
0929 02:58:03.984807 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:03.984813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:03.984885 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:03.984991 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:03.986366 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.987090 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.988333 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:03.988769 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:03.988911 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:03.992536 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:03.992542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:03.992545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:03.992576 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:03.994922 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:03.995550 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:03.995557 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:03.995561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:03.995691 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:03.996058 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:03.996064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:03.996067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:03.996331 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:03.996606 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:03.996876 (+ 270us) server_negotiation.cc:299] Negotiation successful
0929 02:58:03.997091 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:58:04.001286 10147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:58:04.008147 10147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:04.043337 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.027084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.8.208.65:44127)
0929 02:58:04.035564 (+ 8480us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.035571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.036486 (+ 915us) server_negotiation.cc:407] Connection header received
0929 02:58:04.036585 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.036592 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.036799 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.037012 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:04.037550 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.038500 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.040778 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.041997 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.042129 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.042354 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.042842 (+ 488us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.043003 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8339,"thread_start_us":8243,"threads_started":1}
I20260929 02:58:04.043474 10651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.028194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44127 (local address 127.0.0.1:43140)
0929 02:58:04.036205 (+ 8011us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:04.036223 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:04.036333 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:04.036995 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:04.036999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:04.037024 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:04.037381 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.037395 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.038639 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.038644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.040630 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.040642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.042484 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.042497 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.042626 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.042657 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:58:04.042800 (+ 143us) client_negotiation.cc:240] Negotiation successful
0929 02:58:04.043004 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7877,"mutex_wait_us":20,"thread_start_us":8467,"threads_started":1}
I20260929 02:58:04.044283 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.020686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35673 (local address 127.0.0.1:57180)
0929 02:58:04.027790 (+ 7104us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:04.027814 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:04.027950 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:04.035829 (+ 7879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:04.035837 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:04.035898 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:04.036623 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.036636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.038832 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.038840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.041643 (+ 2803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.041652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.043444 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.043459 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.043646 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.043673 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 02:58:04.043732 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 02:58:04.043850 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6946,"mutex_wait_us":231,"spinlock_wait_cycles":2816,"thread_start_us":845,"threads_started":1}
I20260929 02:58:04.044477 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.019745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:38137 (local address 127.0.0.1:38224)
0929 02:58:04.021608 (+ 1863us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:04.021655 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:04.021860 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:04.035829 (+ 13969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:04.035837 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:04.035892 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:04.036629 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.036637 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.038744 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.038752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.041638 (+ 2886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.041652 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.043473 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.043479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.043856 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.043885 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 02:58:04.043955 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 02:58:04.044080 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1521,"mutex_wait_us":367,"spinlock_wait_cycles":384,"thread_start_us":6730,"threads_started":1}
I20260929 02:58:04.045452 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.019997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38224 (local address 127.8.208.66:38137)
0929 02:58:04.035166 (+ 15169us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.035176 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.035201 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:58:04.035301 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.035311 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.035567 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.036847 (+ 1280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:04.037050 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.038567 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.041814 (+ 3247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.043173 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.044738 (+ 1565us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.044938 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.045011 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.045155 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15002,"thread_start_us":15587,"threads_started":1}
I20260929 02:58:04.045962 10649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.021371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57180 (local address 127.8.208.67:35673)
0929 02:58:04.034499 (+ 13128us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.034512 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.034556 (+ 44us) server_negotiation.cc:407] Connection header received
0929 02:58:04.034682 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.034694 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.035033 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.036869 (+ 1836us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:04.037152 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.038697 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.041815 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.043173 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.045260 (+ 2087us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.045428 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.045484 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.045659 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12897,"thread_start_us":14200,"threads_started":1}
I20260929 02:58:04.047374 10285 tablet_service.cc:1467] Processing CreateTablet for tablet 11966b4e69654fd6b47471e108c072f3 (DEFAULT_TABLE table=test-table [id=8c258feb8a404443bfb6c4856823b0d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:04.048580 10285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11966b4e69654fd6b47471e108c072f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:04.048250 10424 tablet_service.cc:1467] Processing CreateTablet for tablet 11966b4e69654fd6b47471e108c072f3 (DEFAULT_TABLE table=test-table [id=8c258feb8a404443bfb6c4856823b0d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:04.049589 10424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11966b4e69654fd6b47471e108c072f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:04.055687 10563 tablet_service.cc:1467] Processing CreateTablet for tablet 11966b4e69654fd6b47471e108c072f3 (DEFAULT_TABLE table=test-table [id=8c258feb8a404443bfb6c4856823b0d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:04.057026 10563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11966b4e69654fd6b47471e108c072f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:04.060715 10652 tablet_bootstrap.cc:492] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48: Bootstrap starting.
I20260929 02:58:04.063644 10652 tablet_bootstrap.cc:654] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:04.064436 10653 tablet_bootstrap.cc:492] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a: Bootstrap starting.
I20260929 02:58:04.064919 10652 log.cc:824] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:04.067780 10653 tablet_bootstrap.cc:654] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:04.068012 10652 tablet_bootstrap.cc:492] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48: No bootstrap required, opened a new log
I20260929 02:58:04.068321 10652 ts_tablet_manager.cc:1397] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260929 02:58:04.069275 10653 log.cc:824] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:04.071125 10656 tablet_bootstrap.cc:492] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3: Bootstrap starting.
I20260929 02:58:04.073987 10656 tablet_bootstrap.cc:654] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:04.074916 10656 log.cc:824] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:04.075822 10653 tablet_bootstrap.cc:492] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a: No bootstrap required, opened a new log
I20260929 02:58:04.076108 10653 ts_tablet_manager.cc:1397] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260929 02:58:04.077329 10656 tablet_bootstrap.cc:492] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3: No bootstrap required, opened a new log
I20260929 02:58:04.077566 10656 ts_tablet_manager.cc:1397] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260929 02:58:04.079984 10652 raft_consensus.cc:348] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.080418 10652 raft_consensus.cc:374] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:04.080526 10652 raft_consensus.cc:729] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8c963293ec54ac28537717e51010e48, State: Initialized, Role: FOLLOWER
I20260929 02:58:04.081108 10652 consensus_queue.cc:260] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.084867 10656 raft_consensus.cc:348] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.085206 10656 raft_consensus.cc:374] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:04.085330 10656 raft_consensus.cc:729] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e62477c4f9140649ffd57d0db5351a3, State: Initialized, Role: FOLLOWER
I20260929 02:58:04.085925 10494 heartbeater.cc:503] Master 127.8.208.126:42781 was elected leader, sending a full tablet report...
I20260929 02:58:04.086695 10652 ts_tablet_manager.cc:1428] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48: Time spent starting tablet: real 0.018s user 0.019s sys 0.000s
I20260929 02:58:04.087030 10656 consensus_queue.cc:260] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.087674 10653 raft_consensus.cc:348] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.088086 10653 raft_consensus.cc:374] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:04.088095 10633 heartbeater.cc:503] Master 127.8.208.126:42781 was elected leader, sending a full tablet report...
I20260929 02:58:04.088178 10653 raft_consensus.cc:729] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b589b5c9994f418c9ea1c570c0c40a, State: Initialized, Role: FOLLOWER
I20260929 02:58:04.088735 10656 ts_tablet_manager.cc:1428] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3: Time spent starting tablet: real 0.011s user 0.005s sys 0.004s
I20260929 02:58:04.088721 10653 consensus_queue.cc:260] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.091655 10653 ts_tablet_manager.cc:1428] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
W20260929 02:58:04.187489 10634 tablet.cc:2369] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:58:04.191084 10356 tablet.cc:2369] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:58:04.195415 10495 tablet.cc:2369] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:58:04.226883 10660 raft_consensus.cc:482] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:04.227247 10660 raft_consensus.cc:504] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.228976 10660 leader_election.cc:288] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8c963293ec54ac28537717e51010e48 (127.8.208.66:38137), 5e62477c4f9140649ffd57d0db5351a3 (127.8.208.67:35673)
I20260929 02:58:04.235739 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.229318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:49753 (local address 127.8.208.66:38137)
0929 02:58:04.229527 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.229534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.230520 (+ 986us) server_negotiation.cc:407] Connection header received
0929 02:58:04.230741 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.230745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.230805 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.230920 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:04.231516 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.232239 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.234108 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.234843 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.235047 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.235121 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.235375 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.235496 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 02:58:04.235824 10649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.229564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:54099 (local address 127.8.208.67:35673)
0929 02:58:04.229800 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.229808 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.230816 (+ 1008us) server_negotiation.cc:407] Connection header received
0929 02:58:04.231002 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.231005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.231060 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.231157 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:04.231727 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.232434 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.234174 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.234925 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.235115 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.235190 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.235456 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.235575 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 02:58:04.235842 10662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.229383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:38137 (local address 127.8.208.65:49753)
0929 02:58:04.230370 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:04.230393 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:04.230576 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:04.230935 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:04.230940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:04.230966 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:04.231372 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.231381 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.232380 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.232386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.233957 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.233968 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.235060 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.235066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.235223 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.235246 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:58:04.235350 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 02:58:04.235481 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":991,"threads_started":1}
I20260929 02:58:04.236111 10661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.229491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35673 (local address 127.8.208.65:54099)
0929 02:58:04.230714 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:04.230737 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:04.230862 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:04.231181 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:04.231185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:04.231208 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:04.231580 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.231588 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.232588 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.232593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.234024 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.234035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.235150 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.235156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.235297 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.235319 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:58:04.235431 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 02:58:04.235554 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1108,"thread_start_us":524,"threads_started":1}
I20260929 02:58:04.237025 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11966b4e69654fd6b47471e108c072f3" candidate_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8c963293ec54ac28537717e51010e48" is_pre_election: true
I20260929 02:58:04.237064 10588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11966b4e69654fd6b47471e108c072f3" candidate_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e62477c4f9140649ffd57d0db5351a3" is_pre_election: true
I20260929 02:58:04.237571 10588 raft_consensus.cc:2454] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b589b5c9994f418c9ea1c570c0c40a in term 0.
I20260929 02:58:04.237565 10449 raft_consensus.cc:2454] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6b589b5c9994f418c9ea1c570c0c40a in term 0.
I20260929 02:58:04.238168 10235 leader_election.cc:302] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e62477c4f9140649ffd57d0db5351a3, f6b589b5c9994f418c9ea1c570c0c40a; no voters:
I20260929 02:58:04.238561 10660 raft_consensus.cc:2781] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:04.238674 10660 raft_consensus.cc:482] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:04.238790 10660 raft_consensus.cc:3037] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:04.240947 10660 raft_consensus.cc:504] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.241605 10660 leader_election.cc:288] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [CANDIDATE]: Term 1 election: Requested vote from peers e8c963293ec54ac28537717e51010e48 (127.8.208.66:38137), 5e62477c4f9140649ffd57d0db5351a3 (127.8.208.67:35673)
I20260929 02:58:04.241958 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11966b4e69654fd6b47471e108c072f3" candidate_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8c963293ec54ac28537717e51010e48"
I20260929 02:58:04.242093 10588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11966b4e69654fd6b47471e108c072f3" candidate_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e62477c4f9140649ffd57d0db5351a3"
I20260929 02:58:04.242177 10449 raft_consensus.cc:3037] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:04.242293 10588 raft_consensus.cc:3037] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:04.245368 10449 raft_consensus.cc:2454] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b589b5c9994f418c9ea1c570c0c40a in term 1.
I20260929 02:58:04.245386 10588 raft_consensus.cc:2454] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6b589b5c9994f418c9ea1c570c0c40a in term 1.
I20260929 02:58:04.245806 10237 leader_election.cc:302] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5e62477c4f9140649ffd57d0db5351a3, e8c963293ec54ac28537717e51010e48, f6b589b5c9994f418c9ea1c570c0c40a; no voters:
I20260929 02:58:04.246155 10660 raft_consensus.cc:2781] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:04.246582 10660 raft_consensus.cc:686] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 1 LEADER]: Becoming Leader. State: Replica: f6b589b5c9994f418c9ea1c570c0c40a, State: Running, Role: LEADER
I20260929 02:58:04.247046 10660 consensus_queue.cc:237] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.251672 10146 catalog_manager.cc:5495] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a reported cstate change: term changed from 0 to 1, leader changed from <none> to f6b589b5c9994f418c9ea1c570c0c40a (127.8.208.65). New cstate: current_term: 1 leader_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } health_report { overall_health: UNKNOWN } } }
I20260929 02:58:04.329226 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.323665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43144 (local address 127.8.208.65:44127)
0929 02:58:04.323906 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.323914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.324050 (+ 136us) server_negotiation.cc:407] Connection header received
0929 02:58:04.324374 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.324378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.324466 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.324611 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:04.324980 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.325741 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.327055 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.327463 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.327600 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.328074 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:04.328546 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:04.328703 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.328837 (+ 134us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.328926 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 02:58:04.336709 10449 raft_consensus.cc:1261] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48 [term 1 FOLLOWER]: Refusing update from remote peer f6b589b5c9994f418c9ea1c570c0c40a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:04.336704 10588 raft_consensus.cc:1261] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3 [term 1 FOLLOWER]: Refusing update from remote peer f6b589b5c9994f418c9ea1c570c0c40a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:04.337683 10660 consensus_queue.cc:1035] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:04.337963 10663 consensus_queue.cc:1035] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:04.366883 10146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:04.367165 10146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:04.370946 10146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076: Sending DeleteTablet for 3 replicas of tablet 11966b4e69654fd6b47471e108c072f3
I20260929 02:58:04.372123 10563 tablet_service.cc:1514] Processing DeleteTablet for tablet 11966b4e69654fd6b47471e108c072f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:04 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:57180
I20260929 02:58:04.372123 10424 tablet_service.cc:1514] Processing DeleteTablet for tablet 11966b4e69654fd6b47471e108c072f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:04 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:38224
I20260929 02:58:04.372133 10285 tablet_service.cc:1514] Processing DeleteTablet for tablet 11966b4e69654fd6b47471e108c072f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:04 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:43140
I20260929 02:58:04.373163 10677 tablet_replica.cc:330] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3: stopping tablet replica
I20260929 02:58:04.373721 10677 raft_consensus.cc:2229] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:04.374326 10677 raft_consensus.cc:2258] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:04.374802 10146 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:58:04.375433 10146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:04.382946 10677 ts_tablet_manager.cc:1905] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:04.383308 10563 tablet_service.cc:1467] Processing CreateTablet for tablet d1feb9c54f4041e2b17c0db53ad6de14 (DEFAULT_TABLE table=test-table [id=a3da40d1970544f1bc143f250e108f2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:04.383740 10563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1feb9c54f4041e2b17c0db53ad6de14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:04.384107 10678 tablet_replica.cc:330] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48: stopping tablet replica
I20260929 02:58:04.384673 10678 raft_consensus.cc:2229] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:04.385272 10678 raft_consensus.cc:2258] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:04.386070 10676 tablet_replica.cc:330] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a: stopping tablet replica
I20260929 02:58:04.386363 10676 raft_consensus.cc:2229] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:04.386842 10676 raft_consensus.cc:2258] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:04.388706 10676 ts_tablet_manager.cc:1905] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:04.390368 10677 ts_tablet_manager.cc:1918] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:04.390520 10677 log.cc:1196] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/11966b4e69654fd6b47471e108c072f3
I20260929 02:58:04.390487 10424 tablet_service.cc:1467] Processing CreateTablet for tablet d1feb9c54f4041e2b17c0db53ad6de14 (DEFAULT_TABLE table=test-table [id=a3da40d1970544f1bc143f250e108f2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:04.390911 10424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1feb9c54f4041e2b17c0db53ad6de14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:04.391031 10677 ts_tablet_manager.cc:1939] T 11966b4e69654fd6b47471e108c072f3 P 5e62477c4f9140649ffd57d0db5351a3: Deleting consensus metadata
I20260929 02:58:04.392695 10132 catalog_manager.cc:4841] TS 5e62477c4f9140649ffd57d0db5351a3 (127.8.208.67:35673): tablet 11966b4e69654fd6b47471e108c072f3 (table test-table [id=8c258feb8a404443bfb6c4856823b0d6]) successfully deleted
I20260929 02:58:04.394634 10678 ts_tablet_manager.cc:1905] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:04.396381 10676 ts_tablet_manager.cc:1918] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:04.396651 10676 log.cc:1196] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/11966b4e69654fd6b47471e108c072f3
I20260929 02:58:04.397322 10676 ts_tablet_manager.cc:1939] T 11966b4e69654fd6b47471e108c072f3 P f6b589b5c9994f418c9ea1c570c0c40a: Deleting consensus metadata
I20260929 02:58:04.397399 10652 tablet_bootstrap.cc:492] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48: Bootstrap starting.
I20260929 02:58:04.399012 10652 tablet_bootstrap.cc:654] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:04.400136 10134 catalog_manager.cc:4841] TS f6b589b5c9994f418c9ea1c570c0c40a (127.8.208.65:44127): tablet 11966b4e69654fd6b47471e108c072f3 (table test-table [id=8c258feb8a404443bfb6c4856823b0d6]) successfully deleted
I20260929 02:58:04.401619 10285 tablet_service.cc:1467] Processing CreateTablet for tablet d1feb9c54f4041e2b17c0db53ad6de14 (DEFAULT_TABLE table=test-table [id=a3da40d1970544f1bc143f250e108f2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:04.402132 10285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1feb9c54f4041e2b17c0db53ad6de14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:04.402253 10656 tablet_bootstrap.cc:492] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3: Bootstrap starting.
I20260929 02:58:04.404350 10656 tablet_bootstrap.cc:654] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:04.409456 10653 tablet_bootstrap.cc:492] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a: Bootstrap starting.
I20260929 02:58:04.411082 10653 tablet_bootstrap.cc:654] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:04.410928 10678 ts_tablet_manager.cc:1918] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:04.411401 10678 log.cc:1196] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/11966b4e69654fd6b47471e108c072f3
I20260929 02:58:04.412048 10678 ts_tablet_manager.cc:1939] T 11966b4e69654fd6b47471e108c072f3 P e8c963293ec54ac28537717e51010e48: Deleting consensus metadata
I20260929 02:58:04.413391 10653 tablet_bootstrap.cc:492] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a: No bootstrap required, opened a new log
I20260929 02:58:04.413595 10653 ts_tablet_manager.cc:1397] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260929 02:58:04.414552 10653 raft_consensus.cc:348] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.414878 10653 raft_consensus.cc:374] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:04.415040 10653 raft_consensus.cc:729] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b589b5c9994f418c9ea1c570c0c40a, State: Initialized, Role: FOLLOWER
I20260929 02:58:04.415376 10653 consensus_queue.cc:260] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.416072 10653 ts_tablet_manager.cc:1428] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:58:04.418749 10656 tablet_bootstrap.cc:492] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3: No bootstrap required, opened a new log
I20260929 02:58:04.418927 10656 ts_tablet_manager.cc:1397] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.004s
I20260929 02:58:04.419765 10656 raft_consensus.cc:348] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.420022 10656 raft_consensus.cc:374] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:04.420131 10656 raft_consensus.cc:729] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e62477c4f9140649ffd57d0db5351a3, State: Initialized, Role: FOLLOWER
I20260929 02:58:04.420447 10656 consensus_queue.cc:260] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.420552 10134 catalog_manager.cc:4841] TS e8c963293ec54ac28537717e51010e48 (127.8.208.66:38137): tablet 11966b4e69654fd6b47471e108c072f3 (table test-table [id=8c258feb8a404443bfb6c4856823b0d6]) successfully deleted
I20260929 02:58:04.421010 10652 tablet_bootstrap.cc:492] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48: No bootstrap required, opened a new log
I20260929 02:58:04.421241 10652 ts_tablet_manager.cc:1397] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.004s
I20260929 02:58:04.421765 10656 ts_tablet_manager.cc:1428] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 02:58:04.422149 10652 raft_consensus.cc:348] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.422420 10652 raft_consensus.cc:374] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:04.422554 10652 raft_consensus.cc:729] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8c963293ec54ac28537717e51010e48, State: Initialized, Role: FOLLOWER
I20260929 02:58:04.422813 10652 consensus_queue.cc:260] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.423435 10652 ts_tablet_manager.cc:1428] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 02:58:04.474038 10659 raft_consensus.cc:482] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:04.474284 10659 raft_consensus.cc:504] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.475885 10659 leader_election.cc:288] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6b589b5c9994f418c9ea1c570c0c40a (127.8.208.65:44127), e8c963293ec54ac28537717e51010e48 (127.8.208.66:38137)
I20260929 02:58:04.483091 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.476070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44127 (local address 127.8.208.67:37873)
0929 02:58:04.476768 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:04.476784 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:04.476942 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:04.477534 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:04.477545 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:04.477580 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:04.477861 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.477874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.479355 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.479360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.480901 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.480912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.482280 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.482286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.482466 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.482502 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 02:58:04.482600 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 02:58:04.482736 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":536,"threads_started":1}
I20260929 02:58:04.483103 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.476392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:37873 (local address 127.8.208.65:44127)
0929 02:58:04.476596 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.476603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.476882 (+ 279us) server_negotiation.cc:407] Connection header received
0929 02:58:04.477119 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.477125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.477242 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.477311 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:04.478048 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.479215 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.481114 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.482118 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.482296 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.482384 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.482647 (+ 263us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.482782 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 02:58:04.484570 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.477297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:45251 (local address 127.8.208.66:38137)
0929 02:58:04.477479 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.477484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.477997 (+ 513us) server_negotiation.cc:407] Connection header received
0929 02:58:04.478219 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.478224 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.478277 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.478377 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:04.478835 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.479735 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.482226 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.483456 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.483619 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.483676 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.484129 (+ 453us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.484282 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:58:04.484572 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.476983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:38137 (local address 127.8.208.67:45251)
0929 02:58:04.477907 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:04.477925 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:04.478081 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:04.478391 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:04.478395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:04.478419 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:04.478678 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.478687 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.479878 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.479884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.482011 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.482028 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.483788 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.483792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.483954 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.483986 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:58:04.484083 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 02:58:04.484213 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":1,"thread_start_us":630,"threads_started":1}
I20260929 02:58:04.484338 10310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1feb9c54f4041e2b17c0db53ad6de14" candidate_uuid: "5e62477c4f9140649ffd57d0db5351a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" is_pre_election: true
I20260929 02:58:04.484856 10310 raft_consensus.cc:2454] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e62477c4f9140649ffd57d0db5351a3 in term 0.
I20260929 02:58:04.485347 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1feb9c54f4041e2b17c0db53ad6de14" candidate_uuid: "5e62477c4f9140649ffd57d0db5351a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8c963293ec54ac28537717e51010e48" is_pre_election: true
I20260929 02:58:04.485522 10516 leader_election.cc:302] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e62477c4f9140649ffd57d0db5351a3, f6b589b5c9994f418c9ea1c570c0c40a; no voters:
I20260929 02:58:04.485598 10449 raft_consensus.cc:2454] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e62477c4f9140649ffd57d0db5351a3 in term 0.
I20260929 02:58:04.485921 10659 raft_consensus.cc:2781] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:04.486053 10659 raft_consensus.cc:482] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:04.486146 10659 raft_consensus.cc:3037] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:04.489444 10659 raft_consensus.cc:504] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.489984 10659 leader_election.cc:288] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [CANDIDATE]: Term 1 election: Requested vote from peers f6b589b5c9994f418c9ea1c570c0c40a (127.8.208.65:44127), e8c963293ec54ac28537717e51010e48 (127.8.208.66:38137)
I20260929 02:58:04.490502 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1feb9c54f4041e2b17c0db53ad6de14" candidate_uuid: "5e62477c4f9140649ffd57d0db5351a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8c963293ec54ac28537717e51010e48"
I20260929 02:58:04.490693 10449 raft_consensus.cc:3037] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:04.493214 10449 raft_consensus.cc:2454] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e62477c4f9140649ffd57d0db5351a3 in term 1.
I20260929 02:58:04.493609 10516 leader_election.cc:302] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e62477c4f9140649ffd57d0db5351a3, e8c963293ec54ac28537717e51010e48; no voters:
I20260929 02:58:04.493623 10310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1feb9c54f4041e2b17c0db53ad6de14" candidate_uuid: "5e62477c4f9140649ffd57d0db5351a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b589b5c9994f418c9ea1c570c0c40a"
I20260929 02:58:04.493820 10310 raft_consensus.cc:3037] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:04.493909 10659 raft_consensus.cc:2781] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:04.494356 10659 raft_consensus.cc:686] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 1 LEADER]: Becoming Leader. State: Replica: 5e62477c4f9140649ffd57d0db5351a3, State: Running, Role: LEADER
I20260929 02:58:04.494772 10659 consensus_queue.cc:237] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.497236 10310 raft_consensus.cc:2454] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e62477c4f9140649ffd57d0db5351a3 in term 1.
I20260929 02:58:04.498559 10147 catalog_manager.cc:5495] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e62477c4f9140649ffd57d0db5351a3 (127.8.208.67). New cstate: current_term: 1 leader_uuid: "5e62477c4f9140649ffd57d0db5351a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } health_report { overall_health: HEALTHY } } }
Sep 29 02:58:04 dist-test-slave-m1rv krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650682, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:04.554831 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.539527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.8.208.126:42781)
0929 02:58:04.540580 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.540585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.540605 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:58:04.540653 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.540656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.540704 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.540815 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:04.542078 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.542886 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.544130 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.544562 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.544870 (+ 308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.548649 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:04.548656 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:04.548661 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:04.548699 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:04.552079 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:04.552782 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:04.552790 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:04.552795 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:04.552937 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:04.553347 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:04.553354 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:04.553357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:04.553610 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:04.553758 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.554149 (+ 391us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.554382 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":920,"thread_start_us":779,"threads_started":1}
I20260929 02:58:04.554878 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.539539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:42781 (local address 127.0.0.1:47538)
0929 02:58:04.540262 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:04.540288 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:04.540472 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:04.540835 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:04.540839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:04.541664 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:04.541926 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.541935 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.543019 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.543024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.543998 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.544008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.544919 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.544925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.545087 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.545819 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:04.545842 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:04.548425 (+ 2583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:04.552255 (+ 3830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:04.552265 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:04.552271 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:04.552663 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:04.553066 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:04.553070 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:04.553072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:04.553249 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:04.553733 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:04.553741 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:04.553890 (+ 149us) client_negotiation.cc:769] Sending connection context
0929 02:58:04.554071 (+ 181us) client_negotiation.cc:240] Negotiation successful
0929 02:58:04.554500 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":161,"thread_start_us":376,"threads_started":1}
I20260929 02:58:04.561558 10147 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:47538:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 02:58:04.562945 10147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:04.570676 10563 tablet_service.cc:1467] Processing CreateTablet for tablet 48f0ee7f24704fbbb62b93b99a5c6246 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d2389e517aab44c2bdd6fbcd18e39ef6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:58:04.570663 10285 tablet_service.cc:1467] Processing CreateTablet for tablet 48f0ee7f24704fbbb62b93b99a5c6246 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d2389e517aab44c2bdd6fbcd18e39ef6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:58:04.570950 10424 tablet_service.cc:1467] Processing CreateTablet for tablet 48f0ee7f24704fbbb62b93b99a5c6246 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d2389e517aab44c2bdd6fbcd18e39ef6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:58:04.571245 10285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48f0ee7f24704fbbb62b93b99a5c6246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:04.571234 10563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48f0ee7f24704fbbb62b93b99a5c6246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:04.571574 10424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48f0ee7f24704fbbb62b93b99a5c6246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:04.578414 10653 tablet_bootstrap.cc:492] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a: Bootstrap starting.
I20260929 02:58:04.578843 10656 tablet_bootstrap.cc:492] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: Bootstrap starting.
I20260929 02:58:04.580075 10656 tablet_bootstrap.cc:654] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:04.580191 10653 tablet_bootstrap.cc:654] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:04.583938 10652 tablet_bootstrap.cc:492] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48: Bootstrap starting.
I20260929 02:58:04.585613 10652 tablet_bootstrap.cc:654] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:04.592739 10656 tablet_bootstrap.cc:492] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: No bootstrap required, opened a new log
I20260929 02:58:04.592954 10656 ts_tablet_manager.cc:1397] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.010s
I20260929 02:58:04.593819 10656 raft_consensus.cc:348] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.594099 10656 raft_consensus.cc:374] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:04.594187 10656 raft_consensus.cc:729] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e62477c4f9140649ffd57d0db5351a3, State: Initialized, Role: FOLLOWER
I20260929 02:58:04.594458 10656 consensus_queue.cc:260] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.594885 10659 tablet_replica.cc:439] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } } }
I20260929 02:58:04.595148 10659 tablet_replica.cc:442] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:04.595184 10653 tablet_bootstrap.cc:492] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a: No bootstrap required, opened a new log
I20260929 02:58:04.595466 10653 ts_tablet_manager.cc:1397] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.001s
I20260929 02:58:04.596730 10653 raft_consensus.cc:348] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.597065 10653 raft_consensus.cc:374] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:04.597218 10653 raft_consensus.cc:729] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b589b5c9994f418c9ea1c570c0c40a, State: Initialized, Role: FOLLOWER
I20260929 02:58:04.597434 10652 tablet_bootstrap.cc:492] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48: No bootstrap required, opened a new log
I20260929 02:58:04.597599 10652 ts_tablet_manager.cc:1397] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260929 02:58:04.597615 10653 consensus_queue.cc:260] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.598364 10656 ts_tablet_manager.cc:1428] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260929 02:58:04.598312 10652 raft_consensus.cc:348] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.598320 10663 tablet_replica.cc:439] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } } }
I20260929 02:58:04.598559 10652 raft_consensus.cc:374] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:04.598646 10652 raft_consensus.cc:729] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8c963293ec54ac28537717e51010e48, State: Initialized, Role: FOLLOWER
I20260929 02:58:04.598645 10663 tablet_replica.cc:442] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:04.598668 10653 ts_tablet_manager.cc:1428] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:04.598958 10652 consensus_queue.cc:260] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.599401 10658 tablet_replica.cc:439] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } } }
I20260929 02:58:04.599740 10658 tablet_replica.cc:442] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:04.601550 10652 ts_tablet_manager.cc:1428] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260929 02:58:04.703963 10659 raft_consensus.cc:482] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:04.704216 10659 raft_consensus.cc:504] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.705013 10659 leader_election.cc:288] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6b589b5c9994f418c9ea1c570c0c40a (127.8.208.65:44127), e8c963293ec54ac28537717e51010e48 (127.8.208.66:38137)
I20260929 02:58:04.705641 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48f0ee7f24704fbbb62b93b99a5c6246" candidate_uuid: "5e62477c4f9140649ffd57d0db5351a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8c963293ec54ac28537717e51010e48" is_pre_election: true
I20260929 02:58:04.705629 10310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48f0ee7f24704fbbb62b93b99a5c6246" candidate_uuid: "5e62477c4f9140649ffd57d0db5351a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" is_pre_election: true
I20260929 02:58:04.705925 10449 raft_consensus.cc:2454] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e62477c4f9140649ffd57d0db5351a3 in term 0.
I20260929 02:58:04.705953 10310 raft_consensus.cc:2454] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e62477c4f9140649ffd57d0db5351a3 in term 0.
I20260929 02:58:04.706408 10516 leader_election.cc:302] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e62477c4f9140649ffd57d0db5351a3, e8c963293ec54ac28537717e51010e48; no voters:
I20260929 02:58:04.706681 10659 raft_consensus.cc:2781] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:04.706780 10659 raft_consensus.cc:482] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:04.706887 10659 raft_consensus.cc:3037] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:04.709225 10659 raft_consensus.cc:504] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.709827 10659 leader_election.cc:288] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [CANDIDATE]: Term 1 election: Requested vote from peers f6b589b5c9994f418c9ea1c570c0c40a (127.8.208.65:44127), e8c963293ec54ac28537717e51010e48 (127.8.208.66:38137)
I20260929 02:58:04.710299 10310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48f0ee7f24704fbbb62b93b99a5c6246" candidate_uuid: "5e62477c4f9140649ffd57d0db5351a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6b589b5c9994f418c9ea1c570c0c40a"
I20260929 02:58:04.710315 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48f0ee7f24704fbbb62b93b99a5c6246" candidate_uuid: "5e62477c4f9140649ffd57d0db5351a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8c963293ec54ac28537717e51010e48"
I20260929 02:58:04.710544 10449 raft_consensus.cc:3037] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:04.710556 10310 raft_consensus.cc:3037] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:04.713629 10449 raft_consensus.cc:2454] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e62477c4f9140649ffd57d0db5351a3 in term 1.
I20260929 02:58:04.713711 10310 raft_consensus.cc:2454] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e62477c4f9140649ffd57d0db5351a3 in term 1.
I20260929 02:58:04.714078 10516 leader_election.cc:302] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e62477c4f9140649ffd57d0db5351a3, e8c963293ec54ac28537717e51010e48; no voters:
I20260929 02:58:04.714444 10659 raft_consensus.cc:2781] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:04.714670 10659 raft_consensus.cc:686] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [term 1 LEADER]: Becoming Leader. State: Replica: 5e62477c4f9140649ffd57d0db5351a3, State: Running, Role: LEADER
I20260929 02:58:04.714965 10659 consensus_queue.cc:237] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } }
I20260929 02:58:04.716632 10684 tablet_replica.cc:439] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: TxnStatusTablet state changed. Reason: New leader 5e62477c4f9140649ffd57d0db5351a3. Latest consensus state: current_term: 1 leader_uuid: "5e62477c4f9140649ffd57d0db5351a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } } }
I20260929 02:58:04.716990 10684 tablet_replica.cc:442] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:58:04.718463 10147 catalog_manager.cc:5495] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e62477c4f9140649ffd57d0db5351a3 (127.8.208.67). New cstate: current_term: 1 leader_uuid: "5e62477c4f9140649ffd57d0db5351a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } health_report { overall_health: HEALTHY } } }
I20260929 02:58:04.719455 10697 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 02:58:04.719682 10697 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 02:58:04.748155 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.739913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35673 (local address 127.0.0.1:57194)
0929 02:58:04.740230 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:04.740250 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:04.740388 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:04.741438 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:04.741443 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:04.741458 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:58:04.741671 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.741682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.743261 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.743266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.744949 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.744965 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.746040 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.746047 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.746246 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.746321 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:58:04.747437 (+ 1116us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:58:04.747465 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 02:58:04.747562 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 02:58:04.747721 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260929 02:58:04.748201 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.739926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57194 (local address 127.8.208.67:35673)
0929 02:58:04.740940 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.740946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.740966 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:58:04.741059 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.741065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.741198 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.741360 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:04.741838 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.743110 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.745113 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.745857 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.745988 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.746532 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:04.747306 (+ 774us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:04.747466 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.747676 (+ 210us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.747766 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":872,"thread_start_us":700,"threads_started":1}
I20260929 02:58:04.752923 10449 raft_consensus.cc:1261] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48 [term 1 FOLLOWER]: Refusing update from remote peer 5e62477c4f9140649ffd57d0db5351a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:04.753242 10310 raft_consensus.cc:1261] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a [term 1 FOLLOWER]: Refusing update from remote peer 5e62477c4f9140649ffd57d0db5351a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:04.753849 10659 consensus_queue.cc:1035] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:04.754233 10684 consensus_queue.cc:1035] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:04.760939 10663 tablet_replica.cc:439] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a: TxnStatusTablet state changed. Reason: New leader 5e62477c4f9140649ffd57d0db5351a3. Latest consensus state: current_term: 1 leader_uuid: "5e62477c4f9140649ffd57d0db5351a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } } }
I20260929 02:58:04.761298 10663 tablet_replica.cc:442] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:04.764165 10658 tablet_replica.cc:439] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48: TxnStatusTablet state changed. Reason: New leader 5e62477c4f9140649ffd57d0db5351a3. Latest consensus state: current_term: 1 leader_uuid: "5e62477c4f9140649ffd57d0db5351a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } } }
I20260929 02:58:04.764470 10658 tablet_replica.cc:442] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:04.766079 10659 tablet_replica.cc:439] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e62477c4f9140649ffd57d0db5351a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } } }
I20260929 02:58:04.766404 10659 tablet_replica.cc:442] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:58:04.766602 10667 mvcc.cc:205] Tried to move back new op lower bound from 7334505204740087808 to 7334505204594524160. Current Snapshot: MvccSnapshot[applied={T|T < 7334505204740087808}]
I20260929 02:58:04.772346 10658 tablet_replica.cc:439] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e62477c4f9140649ffd57d0db5351a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } } }
I20260929 02:58:04.772619 10658 tablet_replica.cc:442] T 48f0ee7f24704fbbb62b93b99a5c6246 P e8c963293ec54ac28537717e51010e48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:04.773643 10684 tablet_replica.cc:439] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e62477c4f9140649ffd57d0db5351a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } } }
I20260929 02:58:04.773923 10684 tablet_replica.cc:442] T 48f0ee7f24704fbbb62b93b99a5c6246 P 5e62477c4f9140649ffd57d0db5351a3: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:58:04.777263 10666 mvcc.cc:205] Tried to move back new op lower bound from 7334505204740087808 to 7334505204594524160. Current Snapshot: MvccSnapshot[applied={T|T < 7334505204740087808}]
I20260929 02:58:04.777595 10663 tablet_replica.cc:439] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e62477c4f9140649ffd57d0db5351a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 } } peers { permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 } } peers { permanent_uuid: "5e62477c4f9140649ffd57d0db5351a3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35673 } } }
I20260929 02:58:04.777853 10663 tablet_replica.cc:442] T 48f0ee7f24704fbbb62b93b99a5c6246 P f6b589b5c9994f418c9ea1c570c0c40a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:04.787420 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.780100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57206 (local address 127.8.208.67:35673)
0929 02:58:04.781253 (+ 1153us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.781259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.781279 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:58:04.781343 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.781350 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.781435 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.781537 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:04.781944 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.783177 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.785136 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.785826 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.785952 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.786476 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:04.786600 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:04.786706 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.786867 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.786945 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1018}
I20260929 02:58:04.806983 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.799268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:53963 (local address 127.8.208.67:35673)
0929 02:58:04.799472 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.799479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.800182 (+ 703us) server_negotiation.cc:407] Connection header received
0929 02:58:04.800377 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.800382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.800489 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.800584 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:04.801225 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.802318 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.804507 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.805240 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.805370 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.805897 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:04.806080 (+ 183us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:04.806207 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.806378 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.806500 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":59}
I20260929 02:58:04.807489 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.799334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35673 (local address 127.8.208.67:53963)
0929 02:58:04.800061 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:04.800086 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:04.800235 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:04.800616 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:04.800621 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:04.800634 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:58:04.801043 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.801055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.802485 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.802490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.804358 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:04.804374 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.805428 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.805435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:04.805628 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.805707 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:58:04.806240 (+ 533us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:58:04.806249 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 02:58:04.806359 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 02:58:04.806585 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":43,"thread_start_us":374,"threads_started":1}
I20260929 02:58:04.819088 10448 raft_consensus.cc:1261] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48 [term 1 FOLLOWER]: Refusing update from remote peer 5e62477c4f9140649ffd57d0db5351a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:04.819437 10310 raft_consensus.cc:1261] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a [term 1 FOLLOWER]: Refusing update from remote peer 5e62477c4f9140649ffd57d0db5351a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:04.819649 10684 consensus_queue.cc:1035] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8c963293ec54ac28537717e51010e48" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 38137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:04.820381 10659 consensus_queue.cc:1035] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6b589b5c9994f418c9ea1c570c0c40a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:04.848357 10668 mvcc.cc:205] Tried to move back new op lower bound from 7334505205012459520 to 7334505203694960640. Current Snapshot: MvccSnapshot[applied={T|T < 7334505205012459520}]
I20260929 02:58:04.908234 10147 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:04.908478 10147 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:04.911652 10147 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df77afafbd25487f917596aa0d08c076: Sending DeleteTablet for 3 replicas of tablet d1feb9c54f4041e2b17c0db53ad6de14
I20260929 02:58:04.912411 10285 tablet_service.cc:1514] Processing DeleteTablet for tablet d1feb9c54f4041e2b17c0db53ad6de14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:04 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:43140
I20260929 02:58:04.912576 10424 tablet_service.cc:1514] Processing DeleteTablet for tablet d1feb9c54f4041e2b17c0db53ad6de14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:04 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:38224
I20260929 02:58:04.913333 10710 tablet_replica.cc:330] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a: stopping tablet replica
I20260929 02:58:04.913532 10710 raft_consensus.cc:2229] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:04.913759 10710 raft_consensus.cc:2258] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:04.913796 10678 tablet_replica.cc:330] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48: stopping tablet replica
I20260929 02:58:04.913986 10678 raft_consensus.cc:2229] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:04.914208 10678 raft_consensus.cc:2258] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:04.914489 10563 tablet_service.cc:1514] Processing DeleteTablet for tablet d1feb9c54f4041e2b17c0db53ad6de14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:04 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:57180
I20260929 02:58:04.915751 10678 ts_tablet_manager.cc:1905] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:04.915769 10710 ts_tablet_manager.cc:1905] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:04.923370 10710 ts_tablet_manager.cc:1918] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:04.923512 10710 log.cc:1196] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/d1feb9c54f4041e2b17c0db53ad6de14
I20260929 02:58:04.923985 10710 ts_tablet_manager.cc:1939] T d1feb9c54f4041e2b17c0db53ad6de14 P f6b589b5c9994f418c9ea1c570c0c40a: Deleting consensus metadata
I20260929 02:58:04.924501 10678 ts_tablet_manager.cc:1918] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:04.925055 10712 tablet_replica.cc:330] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3: stopping tablet replica
I20260929 02:58:04.925246 10134 catalog_manager.cc:4841] TS f6b589b5c9994f418c9ea1c570c0c40a (127.8.208.65:44127): tablet d1feb9c54f4041e2b17c0db53ad6de14 (table test-table [id=a3da40d1970544f1bc143f250e108f2e]) successfully deleted
I20260929 02:58:04.925292 10678 log.cc:1196] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/d1feb9c54f4041e2b17c0db53ad6de14
I20260929 02:58:04.925659 10712 raft_consensus.cc:2229] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:04.926295 10678 ts_tablet_manager.cc:1939] T d1feb9c54f4041e2b17c0db53ad6de14 P e8c963293ec54ac28537717e51010e48: Deleting consensus metadata
I20260929 02:58:04.926988 10712 raft_consensus.cc:2258] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:04.928177 10134 catalog_manager.cc:4841] TS e8c963293ec54ac28537717e51010e48 (127.8.208.66:38137): tablet d1feb9c54f4041e2b17c0db53ad6de14 (table test-table [id=a3da40d1970544f1bc143f250e108f2e]) successfully deleted
I20260929 02:58:04.929261 10712 ts_tablet_manager.cc:1905] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:04.935657 10712 ts_tablet_manager.cc:1918] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:04.935874 10712 log.cc:1196] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/d1feb9c54f4041e2b17c0db53ad6de14
I20260929 02:58:04.936453 10712 ts_tablet_manager.cc:1939] T d1feb9c54f4041e2b17c0db53ad6de14 P 5e62477c4f9140649ffd57d0db5351a3: Deleting consensus metadata
I20260929 02:58:04.937661 10132 catalog_manager.cc:4841] TS 5e62477c4f9140649ffd57d0db5351a3 (127.8.208.67:35673): tablet d1feb9c54f4041e2b17c0db53ad6de14 (table test-table [id=a3da40d1970544f1bc143f250e108f2e]) successfully deleted
Sep 29 02:58:04 dist-test-slave-m1rv krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650683, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:58:04.955287 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.928430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43146 (local address 127.8.208.65:44127)
0929 02:58:04.928644 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.928650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.929658 (+ 1008us) server_negotiation.cc:407] Connection header received
0929 02:58:04.929873 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.929879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.929959 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.930076 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:04.932093 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.933235 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.935355 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.936058 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.936179 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.947610 (+ 11431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:04.947624 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:04.947642 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:04.947688 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:04.951787 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:04.952733 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:04.952743 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:04.952747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:04.952924 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:04.953410 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:04.953415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:04.953418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:04.953908 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:04.954144 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.954692 (+ 548us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.954928 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260929 02:58:04.956990 10350 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43146
I20260929 02:58:04.983816 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.960729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.8.208.126:42781)
0929 02:58:04.960947 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.960953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.967654 (+ 6701us) server_negotiation.cc:407] Connection header received
0929 02:58:04.968106 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.968112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.968192 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.968256 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:04.970288 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.971585 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.973667 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.974298 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.974391 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.977479 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:04.977486 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:04.977491 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:04.977528 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:04.980791 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:04.981614 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:04.981619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:04.981623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:04.981754 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:04.982171 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:04.982177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:04.982180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:04.982516 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:04.982723 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:04.983285 (+ 562us) server_negotiation.cc:299] Negotiation successful
0929 02:58:04.983503 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260929 02:58:04.985397 10147 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47550
I20260929 02:58:04.987928 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 10219
I20260929 02:58:05.001710 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:04.989774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43154 (local address 127.8.208.65:44127)
0929 02:58:04.989987 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:04.989992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:04.990381 (+ 389us) server_negotiation.cc:407] Connection header received
0929 02:58:04.990564 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:04.990569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:04.990640 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:04.990748 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:04.991809 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.992517 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.993774 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:04.994243 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:04.994395 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:04.996508 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:04.996514 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:04.996518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:04.996556 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:04.999208 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:04.999831 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:04.999838 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:04.999841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:04.999973 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:05.000321 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:05.000327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:05.000330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:05.000592 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:05.000832 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:05.001124 (+ 292us) server_negotiation.cc:299] Negotiation successful
0929 02:58:05.001371 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260929 02:58:05.002446 10350 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43154
W20260929 02:58:05.003044 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 10219: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:58:05.003257 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 10219
I20260929 02:58:05.031982 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 10359
Sep 29 02:58:05 dist-test-slave-m1rv krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650683, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 02:58:05.048864 10717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:05.032770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38234 (local address 127.8.208.66:38137)
0929 02:58:05.033279 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:05.033283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:05.033299 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:58:05.033352 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:05.033356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:05.033420 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:05.033515 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:05.035108 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:05.035794 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:05.037672 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:05.038054 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:05.038155 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:05.043171 (+ 5016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:05.043183 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:05.043196 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:05.043226 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:05.046164 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:05.046836 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:05.046843 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:05.046845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:05.046969 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:05.047347 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:05.047354 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:05.047357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:05.047694 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:05.047985 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:05.048278 (+ 293us) server_negotiation.cc:299] Negotiation successful
0929 02:58:05.048509 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":268,"threads_started":1}
W20260929 02:58:05.049693 10489 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38234
W20260929 02:58:05.050381 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 10359: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:58:05.050561 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 10359
I20260929 02:58:05.067945 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 10498
Sep 29 02:58:05 dist-test-slave-m1rv krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650683, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 02:58:05.084918 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:05.068877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57216 (local address 127.8.208.67:35673)
0929 02:58:05.069107 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:05.069113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:05.069136 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:58:05.069233 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:05.069238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:05.069315 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:05.069435 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:05.070668 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:05.071887 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:05.073291 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:05.073967 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:05.074098 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:05.078111 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:05.078123 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:05.078141 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:05.078202 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:05.082152 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:05.082768 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:05.082777 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:05.082780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:05.082993 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:05.083341 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:05.083347 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:05.083350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:05.083804 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:05.084146 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:05.084316 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 02:58:05.084536 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260929 02:58:05.085795 10628 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57216
W20260929 02:58:05.086542 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 10498: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:58:05.086694 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 10498
I20260929 02:58:05.102056 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 10116
I20260929 02:58:05.103282 10188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:05.218632 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 10116
2026-09-29T02:58:05Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4594 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:58:05 dist-test-slave-m1rv krb5kdc[10721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:58:05 dist-test-slave-m1rv krb5kdc[10721](info): set up 2 sockets
Sep 29 02:58:05 dist-test-slave-m1rv krb5kdc[10721](info): commencing operation
krb5kdc: starting...
W20260929 02:58:06.305537 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:58:06 dist-test-slave-m1rv krb5kdc[10721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:58:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:58:06.484606 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:33131
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:33131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:06.802358 10734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:06.802723 10734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:06.802906 10734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:06.810155 10734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:06.810305 10734 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 02:58:06.810386 10734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:06.810493 10734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:06.810572 10734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:06.810659 10734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:06.820472 10734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:33131
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:33131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:06.821796 10734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:06.823112 10734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:06.834885 10740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:06.834991 10739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:06.835490 10734 server_base.cc:1034] running on GCE node
W20260929 02:58:06.835547 10742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:06.836351 10734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:06.837668 10734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:06.838874 10734 hybrid_clock.cc:648] HybridClock initialized: now 1790650686838819 us; error 34 us; skew 500 ppm
I20260929 02:58:06.839183 10734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:06 dist-test-slave-m1rv krb5kdc[10721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650686, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:06.847962 10734 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:58:06.849429 10734 webserver.cc:460] Webserver started at http://127.8.208.126:33981/ using document root <none> and password file <none>
I20260929 02:58:06.850133 10734 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:06.850263 10734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:06.850579 10734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:06.853546 10734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "dc8c58f7bb864ca89f807896daef44e4"
format_stamp: "Formatted at 2026-09-29 02:58:06 on dist-test-slave-m1rv"
server_key: "8c64473b3a9b39310522642ce5a4527d"
server_key_iv: "0ac990c4a13edec778b95a1568ff014b"
server_key_version: "encryptionkey@0"
I20260929 02:58:06.854293 10734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "dc8c58f7bb864ca89f807896daef44e4"
format_stamp: "Formatted at 2026-09-29 02:58:06 on dist-test-slave-m1rv"
server_key: "8c64473b3a9b39310522642ce5a4527d"
server_key_iv: "0ac990c4a13edec778b95a1568ff014b"
server_key_version: "encryptionkey@0"
I20260929 02:58:06.859277 10734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:58:06.862619 10749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:06.863860 10734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:58:06.864034 10734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "dc8c58f7bb864ca89f807896daef44e4"
format_stamp: "Formatted at 2026-09-29 02:58:06 on dist-test-slave-m1rv"
server_key: "8c64473b3a9b39310522642ce5a4527d"
server_key_iv: "0ac990c4a13edec778b95a1568ff014b"
server_key_version: "encryptionkey@0"
I20260929 02:58:06.864241 10734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:06.878846 10734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:06.882817 10734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:06.883081 10734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:06.904382 10734 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:33131
I20260929 02:58:06.904407 10810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:33131 every 8 connection(s)
I20260929 02:58:06.905817 10734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:06.909469 10811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:06.911559 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 10734
I20260929 02:58:06.911772 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:06.912552 9025 external_mini_cluster.cc:1428] Setting key a64e6d1110b1131b2f084e06cf8e7857
I20260929 02:58:06.922091 10811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4: Bootstrap starting.
I20260929 02:58:06.925930 10811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4: Neither blocks nor log segments found. Creating new log.
Sep 29 02:58:06 dist-test-slave-m1rv krb5kdc[10721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:06.927126 10811 log.cc:824] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:06.930094 10811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4: No bootstrap required, opened a new log
I20260929 02:58:06.935417 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:06.914617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52728 (local address 127.8.208.126:33131)
0929 02:58:06.915457 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:06.915471 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:06.915515 (+ 44us) server_negotiation.cc:407] Connection header received
0929 02:58:06.917295 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:06.917319 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:06.917704 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:06.918052 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:06.919769 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:06.921078 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:06.922729 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:06.923371 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:06.923518 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:06.928124 (+ 4606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:06.928131 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:06.928149 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:06.928187 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:06.931315 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:06.932010 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:06.932017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:06.932023 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:06.932206 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:06.932580 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:06.932585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:06.932593 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:06.933098 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:06.933442 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:06.933904 (+ 462us) server_negotiation.cc:299] Negotiation successful
0929 02:58:06.934339 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":414,"threads_started":1}
I20260929 02:58:06.938040 10811 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc8c58f7bb864ca89f807896daef44e4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33131 } }
I20260929 02:58:06.938491 10811 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:06.938601 10811 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc8c58f7bb864ca89f807896daef44e4, State: Initialized, Role: FOLLOWER
I20260929 02:58:06.939189 10811 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc8c58f7bb864ca89f807896daef44e4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33131 } }
I20260929 02:58:06.939417 10811 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:06.939559 10811 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:06.939736 10811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:06.942155 10811 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc8c58f7bb864ca89f807896daef44e4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33131 } }
I20260929 02:58:06.942654 10811 leader_election.cc:302] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc8c58f7bb864ca89f807896daef44e4; no voters:
I20260929 02:58:06.943213 10811 leader_election.cc:288] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:06.943336 10816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:06.944023 10816 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [term 1 LEADER]: Becoming Leader. State: Replica: dc8c58f7bb864ca89f807896daef44e4, State: Running, Role: LEADER
I20260929 02:58:06.944567 10816 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc8c58f7bb864ca89f807896daef44e4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33131 } }
I20260929 02:58:06.945261 10811 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:06.947424 10817 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc8c58f7bb864ca89f807896daef44e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc8c58f7bb864ca89f807896daef44e4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33131 } } }
I20260929 02:58:06.948403 10817 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:06.949507 10818 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc8c58f7bb864ca89f807896daef44e4. Latest consensus state: current_term: 1 leader_uuid: "dc8c58f7bb864ca89f807896daef44e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc8c58f7bb864ca89f807896daef44e4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33131 } } }
I20260929 02:58:06.949903 10818 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:06.955024 10824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:06.960850 10824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:06.969260 10824 catalog_manager.cc:1347] Generated new cluster ID: f054e13be63e474ab35fe17b1128295e
I20260929 02:58:06.969369 10824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:06.976660 10824 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:06.978502 10824 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:06.994488 10824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4: Generated new TSK 0
I20260929 02:58:06.995522 10824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 02:58:07.044404 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:33131
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:07.373026 10837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:07.373390 10837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:07.373570 10837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:07.380712 10837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:07.380847 10837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:07.380991 10837 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 02:58:07.381073 10837 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:58:07.381217 10837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:58:07.390807 10837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.208.126:33131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:07.392027 10837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:07.393392 10837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:07.406258 10845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:07.406279 10843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:07.406483 10842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:07.407203 10837 server_base.cc:1034] running on GCE node
I20260929 02:58:07.407770 10837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:07.408633 10837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:07.409823 10837 hybrid_clock.cc:648] HybridClock initialized: now 1790650687409785 us; error 36 us; skew 500 ppm
I20260929 02:58:07.410147 10837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:07 dist-test-slave-m1rv krb5kdc[10721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650687, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:07.413302 10837 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 02:58:07.414630 10837 webserver.cc:460] Webserver started at http://127.8.208.65:39397/ using document root <none> and password file <none>
I20260929 02:58:07.415241 10837 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:07.415342 10837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:07.415627 10837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:07.418838 10837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "35dd9a6f086842cbb07f628dc0b94feb"
format_stamp: "Formatted at 2026-09-29 02:58:07 on dist-test-slave-m1rv"
server_key: "255a9b8b51e70fe66e1e41cf5155741b"
server_key_iv: "7f894e0d946eb3f24616b0c7594247f7"
server_key_version: "encryptionkey@0"
I20260929 02:58:07.419593 10837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "35dd9a6f086842cbb07f628dc0b94feb"
format_stamp: "Formatted at 2026-09-29 02:58:07 on dist-test-slave-m1rv"
server_key: "255a9b8b51e70fe66e1e41cf5155741b"
server_key_iv: "7f894e0d946eb3f24616b0c7594247f7"
server_key_version: "encryptionkey@0"
I20260929 02:58:07.424499 10837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:58:07.427708 10852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:07.428982 10837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:58:07.429164 10837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "35dd9a6f086842cbb07f628dc0b94feb"
format_stamp: "Formatted at 2026-09-29 02:58:07 on dist-test-slave-m1rv"
server_key: "255a9b8b51e70fe66e1e41cf5155741b"
server_key_iv: "7f894e0d946eb3f24616b0c7594247f7"
server_key_version: "encryptionkey@0"
I20260929 02:58:07.429423 10837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:07.469866 10837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:07.474575 10837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:07.474896 10837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:07.477634 10837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:07.477777 10837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:07.477912 10837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:07.477988 10837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:58:07 dist-test-slave-m1rv krb5kdc[10721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650687, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:07.506851 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:07.478639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:46455 (local address 127.8.208.126:33131)
0929 02:58:07.482597 (+ 3958us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:07.482605 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:07.483583 (+ 978us) server_negotiation.cc:407] Connection header received
0929 02:58:07.486775 (+ 3192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:07.486783 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:07.486884 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:07.487024 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:07.488573 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.489731 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.492726 (+ 2995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.493329 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.493464 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:07.498291 (+ 4827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:07.498300 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:07.498307 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:07.498355 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:07.502280 (+ 3925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:07.503491 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:07.503499 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:07.503505 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:07.503674 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:07.504305 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:07.504313 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:07.504319 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:07.504740 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:07.504894 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:07.506105 (+ 1211us) server_negotiation.cc:299] Negotiation successful
0929 02:58:07.506366 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3799,"thread_start_us":3675,"threads_started":1}
I20260929 02:58:07.508811 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:07.479324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33131 (local address 127.8.208.65:46455)
0929 02:58:07.482708 (+ 3384us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:07.482759 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:07.486314 (+ 3555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:07.487282 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:07.487291 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:07.487812 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:07.488393 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:07.488410 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.489876 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.489883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:07.492488 (+ 2605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:07.492507 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.493562 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.493570 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:07.493802 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:07.494661 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:07.494701 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:07.498006 (+ 3305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:07.502478 (+ 4472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:07.502491 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:07.502521 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:07.503339 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:07.503830 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:07.503836 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:07.503840 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:07.504164 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:07.504907 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:07.504919 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:07.505570 (+ 651us) client_negotiation.cc:769] Sending connection context
0929 02:58:07.506019 (+ 449us) client_negotiation.cc:240] Negotiation successful
0929 02:58:07.506758 (+ 739us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2466,"thread_start_us":2236,"threads_started":1}
I20260929 02:58:07.519611 10837 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:41035
I20260929 02:58:07.519634 10972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:41035 every 8 connection(s)
I20260929 02:58:07.521081 10837 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:58:07.523204 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 10837
I20260929 02:58:07.523422 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:58:07.523427 10973 heartbeater.cc:347] Connected to a master server at 127.8.208.126:33131
I20260929 02:58:07.523675 10973 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:07.524235 9025 external_mini_cluster.cc:1428] Setting key 0f70b1a17bcd25cc44346be57b7f5e31
I20260929 02:58:07.524520 10973 heartbeater.cc:511] Master 127.8.208.126:33131 requested a full tablet report, sending...
I20260929 02:58:07.526902 10765 ts_manager.cc:194] Registered new tserver with Master: 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035)
I20260929 02:58:07.528687 10765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:46455
I20260929 02:58:07.530817 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:07.516502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:34423 (local address 127.8.208.126:33131)
0929 02:58:07.516743 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:07.516753 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:07.517370 (+ 617us) server_negotiation.cc:407] Connection header received
0929 02:58:07.517622 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:07.517628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:07.517712 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:07.517808 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:07.518996 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.520135 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.521620 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.522262 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.522403 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:07.524362 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:07.524367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:07.524370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:07.524400 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:07.527794 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:07.528622 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:07.528628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:07.528631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:07.528792 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:07.529275 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:07.529281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:07.529285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:07.529615 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:07.529778 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:07.530273 (+ 495us) server_negotiation.cc:299] Negotiation successful
0929 02:58:07.530488 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:58:07.531323 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:07.516601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33131 (local address 127.8.208.65:34423)
0929 02:58:07.517249 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:07.517272 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:07.517462 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:07.517850 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:07.517853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:07.518395 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:07.518817 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:07.518836 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.520306 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.520310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:07.521439 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:07.521451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.522646 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.522651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:07.522806 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:07.523459 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:07.523476 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:07.524180 (+ 704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:07.527933 (+ 3753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:07.527941 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:07.527947 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:07.528496 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:07.528956 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:07.528963 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:07.528969 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:07.529128 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:07.529756 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:07.529764 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:07.529978 (+ 214us) client_negotiation.cc:769] Sending connection context
0929 02:58:07.530129 (+ 151us) client_negotiation.cc:240] Negotiation successful
0929 02:58:07.530535 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":22,"thread_start_us":327,"threads_started":1}
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 02:58:07.565964 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:33131
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:07.882519 10977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:07.882808 10977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:07.882967 10977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:07.890280 10977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:07.890434 10977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:07.890574 10977 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 02:58:07.890656 10977 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:58:07.890779 10977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:58:07.900465 10977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.208.126:33131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:07.901895 10977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:07.903191 10977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:07.916772 10985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:07.916766 10983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:07.917073 10982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:07.917873 10977 server_base.cc:1034] running on GCE node
I20260929 02:58:07.918413 10977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:07.919348 10977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:07.920567 10977 hybrid_clock.cc:648] HybridClock initialized: now 1790650687920525 us; error 37 us; skew 500 ppm
I20260929 02:58:07.920825 10977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:07 dist-test-slave-m1rv krb5kdc[10721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650687, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:07.923981 10977 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 02:58:07.925400 10977 webserver.cc:460] Webserver started at http://127.8.208.66:39717/ using document root <none> and password file <none>
I20260929 02:58:07.926065 10977 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:07.926189 10977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:07.926496 10977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:07.929334 10977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "93235c192b3d43f0923782e8288ce61a"
format_stamp: "Formatted at 2026-09-29 02:58:07 on dist-test-slave-m1rv"
server_key: "17a54f8c8d939b53c702e819f4aa9fdd"
server_key_iv: "2073ade69fc84c217c3e8260341ae7e0"
server_key_version: "encryptionkey@0"
I20260929 02:58:07.930075 10977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "93235c192b3d43f0923782e8288ce61a"
format_stamp: "Formatted at 2026-09-29 02:58:07 on dist-test-slave-m1rv"
server_key: "17a54f8c8d939b53c702e819f4aa9fdd"
server_key_iv: "2073ade69fc84c217c3e8260341ae7e0"
server_key_version: "encryptionkey@0"
I20260929 02:58:07.935561 10977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:58:07.938961 10992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:07.940321 10977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:58:07.940516 10977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "93235c192b3d43f0923782e8288ce61a"
format_stamp: "Formatted at 2026-09-29 02:58:07 on dist-test-slave-m1rv"
server_key: "17a54f8c8d939b53c702e819f4aa9fdd"
server_key_iv: "2073ade69fc84c217c3e8260341ae7e0"
server_key_version: "encryptionkey@0"
I20260929 02:58:07.940728 10977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:07.954087 10977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:07.958352 10977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:07.958654 10977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:07.961162 10977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:07.961371 10977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:07.961505 10977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:07.961598 10977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:58:07 dist-test-slave-m1rv krb5kdc[10721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650687, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:07.986485 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:07.961910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:43119 (local address 127.8.208.126:33131)
0929 02:58:07.962148 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:07.962158 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:07.963611 (+ 1453us) server_negotiation.cc:407] Connection header received
0929 02:58:07.966663 (+ 3052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:07.966669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:07.966782 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:07.966910 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:07.969504 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.970231 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.973419 (+ 3188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.974009 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.974156 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:07.978707 (+ 4551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:07.978716 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:07.978723 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:07.978779 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:07.982607 (+ 3828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:07.983583 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:07.983589 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:07.983593 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:07.983714 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:07.984340 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:07.984345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:07.984348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:07.984738 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:07.984890 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:07.985914 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0929 02:58:07.986133 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:58:07.987486 11001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:07.962482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33131 (local address 127.8.208.66:43119)
0929 02:58:07.963425 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:07.963481 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:07.966319 (+ 2838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:07.967217 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:07.967234 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:07.968209 (+ 975us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:07.969324 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:07.969358 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.970396 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.970404 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:07.973250 (+ 2846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:07.973270 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.974276 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.974286 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:07.974639 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:07.975501 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:07.975527 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:07.978435 (+ 2908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:07.982771 (+ 4336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:07.982781 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:07.982792 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:07.983455 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:07.983823 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:07.983828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:07.983835 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:07.984180 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:07.984950 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:07.984956 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:07.985491 (+ 535us) client_negotiation.cc:769] Sending connection context
0929 02:58:07.985806 (+ 315us) client_negotiation.cc:240] Negotiation successful
0929 02:58:07.986326 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":338,"threads_started":1}
I20260929 02:58:08.010066 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:07.993733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:38107 (local address 127.8.208.126:33131)
0929 02:58:07.993902 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:07.993909 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:07.994628 (+ 719us) server_negotiation.cc:407] Connection header received
0929 02:58:07.995034 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:07.995039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:07.995116 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:07.995211 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:07.996636 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.997420 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.999427 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.000205 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.000367 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.003286 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:08.003293 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:08.003300 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:08.003357 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:08.007233 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:08.008093 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:08.008100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:08.008104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:08.008236 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:08.008576 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:08.008582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:08.008586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:08.008926 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:08.009084 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.009513 (+ 429us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.009735 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:58:08.010337 11075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:07.993805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33131 (local address 127.8.208.66:38107)
0929 02:58:07.994503 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:07.994527 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:07.994755 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:07.995266 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:07.995274 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:07.996005 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:07.996467 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:07.996483 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:07.997568 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:07.997574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:07.999257 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:07.999275 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.000654 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.000664 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.000920 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.001639 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:08.001665 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:08.003026 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:08.007356 (+ 4330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:08.007361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:08.007365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:08.007972 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:08.008343 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:08.008348 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:08.008352 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:08.008476 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:08.009042 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:08.009050 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:08.009281 (+ 231us) client_negotiation.cc:769] Sending connection context
0929 02:58:08.009432 (+ 151us) client_negotiation.cc:240] Negotiation successful
0929 02:58:08.009813 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":343,"threads_started":1}
I20260929 02:58:08.012283 10977 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:40093
I20260929 02:58:08.012369 11111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:40093 every 8 connection(s)
I20260929 02:58:08.013664 10977 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:58:08.015698 11112 heartbeater.cc:347] Connected to a master server at 127.8.208.126:33131
I20260929 02:58:08.015878 11112 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:08.016453 11112 heartbeater.cc:511] Master 127.8.208.126:33131 requested a full tablet report, sending...
I20260929 02:58:08.017664 10765 ts_manager.cc:194] Registered new tserver with Master: 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093)
I20260929 02:58:08.018417 10765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:43119
I20260929 02:58:08.023686 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 10977
I20260929 02:58:08.023825 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:58:08.024212 9025 external_mini_cluster.cc:1428] Setting key 3d8f65a6a7b9b179ed28c233de80b5f7
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 02:58:08.064589 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:33131
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:45943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:08.376514 11116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:08.376785 11116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:08.376945 11116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:08.384125 11116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:08.384224 11116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:08.384356 11116 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 02:58:08.384423 11116 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:58:08.384524 11116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:58:08.394318 11116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:45943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.208.126:33131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:08.395561 11116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:08.396869 11116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:08.410125 11121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:08.410132 11122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:08.410784 11116 server_base.cc:1034] running on GCE node
W20260929 02:58:08.410827 11124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:08.411435 11116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:08.412384 11116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:08.413606 11116 hybrid_clock.cc:648] HybridClock initialized: now 1790650688413565 us; error 53 us; skew 500 ppm
I20260929 02:58:08.413903 11116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:08 dist-test-slave-m1rv krb5kdc[10721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650688, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:08.417023 11116 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 02:58:08.418509 11116 webserver.cc:460] Webserver started at http://127.8.208.67:42741/ using document root <none> and password file <none>
I20260929 02:58:08.419133 11116 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:08.419245 11116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:08.419497 11116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:08.422353 11116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "802a4d73449747d69f6c3cacbf95886b"
format_stamp: "Formatted at 2026-09-29 02:58:08 on dist-test-slave-m1rv"
server_key: "8c53ebd0d383a572ec144811d6c2e792"
server_key_iv: "3b3a01bca3328eb551643cd0c64d52ef"
server_key_version: "encryptionkey@0"
I20260929 02:58:08.423024 11116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "802a4d73449747d69f6c3cacbf95886b"
format_stamp: "Formatted at 2026-09-29 02:58:08 on dist-test-slave-m1rv"
server_key: "8c53ebd0d383a572ec144811d6c2e792"
server_key_iv: "3b3a01bca3328eb551643cd0c64d52ef"
server_key_version: "encryptionkey@0"
I20260929 02:58:08.428004 11116 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260929 02:58:08.431239 11131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:08.432586 11116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:08.432714 11116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "802a4d73449747d69f6c3cacbf95886b"
format_stamp: "Formatted at 2026-09-29 02:58:08 on dist-test-slave-m1rv"
server_key: "8c53ebd0d383a572ec144811d6c2e792"
server_key_iv: "3b3a01bca3328eb551643cd0c64d52ef"
server_key_version: "encryptionkey@0"
I20260929 02:58:08.432885 11116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:08.460147 11116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:08.464473 11116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:08.464716 11116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:08.467422 11116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:08.467538 11116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:08.467718 11116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:08.467808 11116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:58:08 dist-test-slave-m1rv krb5kdc[10721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650688, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:08.492048 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.467869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:59509 (local address 127.8.208.126:33131)
0929 02:58:08.468095 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:08.468104 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:08.469589 (+ 1485us) server_negotiation.cc:407] Connection header received
0929 02:58:08.471762 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:08.471767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:08.471863 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:08.471997 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:08.474146 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.475508 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.478166 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.478923 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.479086 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.484218 (+ 5132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:08.484228 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:08.484236 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:08.484294 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:08.488249 (+ 3955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:08.489300 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:08.489308 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:08.489314 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:08.489457 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:08.490131 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:08.490139 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:08.490141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:08.490510 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:08.490666 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.491446 (+ 780us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.491693 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:58:08.493096 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.468530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33131 (local address 127.8.208.67:59509)
0929 02:58:08.469449 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:08.469488 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:08.471476 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:08.472243 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:08.472251 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:08.472941 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:08.473817 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.473837 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.475726 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.475732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.478009 (+ 2277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.478031 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.479243 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.479251 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.479466 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.480434 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:08.480461 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:08.483885 (+ 3424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:08.488403 (+ 4518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:08.488413 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:08.488434 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:08.489148 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:08.489630 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:08.489637 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:08.489642 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:08.490000 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:08.490643 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:08.490651 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:08.491051 (+ 400us) client_negotiation.cc:769] Sending connection context
0929 02:58:08.491320 (+ 269us) client_negotiation.cc:240] Negotiation successful
0929 02:58:08.491873 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":385,"threads_started":1}
I20260929 02:58:08.518791 11213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.504339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33131 (local address 127.8.208.67:60097)
0929 02:58:08.505691 (+ 1352us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:08.505715 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:08.505894 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:08.506297 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:08.506305 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:08.506777 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:08.507059 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.507069 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.508061 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.508066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.509072 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.509083 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.510192 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.510200 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.510450 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.511279 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:08.511298 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:08.512552 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:08.515930 (+ 3378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:08.515934 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:08.515938 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:08.516436 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:08.516822 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:08.516827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:08.516831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:08.516965 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:08.517542 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:08.517548 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:08.517748 (+ 200us) client_negotiation.cc:769] Sending connection context
0929 02:58:08.517912 (+ 164us) client_negotiation.cc:240] Negotiation successful
0929 02:58:08.518281 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1184,"thread_start_us":1005,"threads_started":1}
I20260929 02:58:08.518594 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.504718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:60097 (local address 127.8.208.126:33131)
0929 02:58:08.505040 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:08.505046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:08.505825 (+ 779us) server_negotiation.cc:407] Connection header received
0929 02:58:08.506073 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:08.506079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:08.506166 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:08.506267 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:08.507200 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.507919 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.509229 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.509760 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.509901 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.512812 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:08.512817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:08.512820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:08.512851 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:08.515802 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:08.516576 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:08.516581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:08.516585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:08.516713 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:08.517069 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:08.517075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:08.517078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:08.517423 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:08.517600 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.517992 (+ 392us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.518213 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260929 02:58:08.525825 11116 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:34935
I20260929 02:58:08.525856 11250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:34935 every 8 connection(s)
I20260929 02:58:08.527010 11116 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:58:08.528885 11251 heartbeater.cc:347] Connected to a master server at 127.8.208.126:33131
I20260929 02:58:08.529099 11251 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:08.529748 11251 heartbeater.cc:511] Master 127.8.208.126:33131 requested a full tablet report, sending...
I20260929 02:58:08.531019 10765 ts_manager.cc:194] Registered new tserver with Master: 802a4d73449747d69f6c3cacbf95886b (127.8.208.67:34935)
I20260929 02:58:08.531785 10765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:59509
I20260929 02:58:08.532110 10973 heartbeater.cc:503] Master 127.8.208.126:33131 was elected leader, sending a full tablet report...
I20260929 02:58:08.534336 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 11116
I20260929 02:58:08.534507 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:58:08.534924 9025 external_mini_cluster.cc:1428] Setting key a679c1faf9a98f58c63e623bfce8cdb8
I20260929 02:58:08.537521 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:58:08 dist-test-slave-m1rv krb5kdc[10721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:58:08 dist-test-slave-m1rv krb5kdc[10721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:08.582441 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.569027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60064 (local address 127.8.208.126:33131)
0929 02:58:08.569282 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:08.569288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:08.569499 (+ 211us) server_negotiation.cc:407] Connection header received
0929 02:58:08.569707 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:08.569713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:08.569790 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:08.569897 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:08.571260 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.572017 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.573237 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.573661 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.573795 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.577458 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:08.577463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:08.577466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:08.577499 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:08.579901 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:08.580516 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:08.580520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:08.580523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:08.580643 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:08.581011 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:08.581016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:08.581019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:08.581332 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:08.581578 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.581858 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.582082 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 02:58:08.586287 10765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:58:08.593371 10765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:08.616355 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.605545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35306 (local address 127.8.208.67:34935)
0929 02:58:08.608818 (+ 3273us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:08.608826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:08.608859 (+ 33us) server_negotiation.cc:407] Connection header received
0929 02:58:08.608955 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:08.608963 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:08.609216 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:08.609385 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:08.610540 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.611799 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.614243 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.615343 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.615467 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.615692 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.615865 (+ 173us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.616027 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3101,"thread_start_us":3691,"threads_started":1}
I20260929 02:58:08.616425 11268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.604988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:34935 (local address 127.0.0.1:35306)
0929 02:58:08.607686 (+ 2698us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:08.607724 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:08.608250 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:08.609475 (+ 1225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:08.609483 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:08.609528 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:08.610359 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.610371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.611959 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.611965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.614096 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.614107 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.615515 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.615519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.615645 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.615678 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:58:08.615776 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 02:58:08.615959 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2396,"mutex_wait_us":214,"spinlock_wait_cycles":5760,"thread_start_us":4111,"threads_started":1}
I20260929 02:58:08.617676 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.605279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:40093 (local address 127.0.0.1:41074)
0929 02:58:08.607907 (+ 2628us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:08.607922 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:08.608490 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:08.609475 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:08.609483 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:08.609519 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:08.610216 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.610235 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.612188 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.612193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.614861 (+ 2668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.614876 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.616855 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.616861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.617033 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.617056 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:58:08.617156 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 02:58:08.617304 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2199,"mutex_wait_us":164,"spinlock_wait_cycles":846336,"thread_start_us":2058,"threads_started":1}
I20260929 02:58:08.619064 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.606917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:41035 (local address 127.0.0.1:56856)
0929 02:58:08.608828 (+ 1911us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:08.608846 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:08.608978 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:08.609748 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:08.609754 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:08.609781 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:08.610294 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.610306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.612189 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.612193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.614861 (+ 2668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.614876 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.618415 (+ 3539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.618421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.618584 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.618621 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 02:58:08.618712 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 02:58:08.618803 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1798,"mutex_wait_us":154,"thread_start_us":681,"threads_started":1}
I20260929 02:58:08.619228 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.607482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56856 (local address 127.8.208.65:41035)
0929 02:58:08.608414 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:08.608424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:08.609093 (+ 669us) server_negotiation.cc:407] Connection header received
0929 02:58:08.609214 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:08.609221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:08.609434 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:08.609608 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:08.610521 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.612020 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.616796 (+ 4776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.618227 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.618387 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.618583 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.618789 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.618963 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":661,"threads_started":1}
I20260929 02:58:08.620389 11181 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1b12a8695d4414a18688da5b354a92 (DEFAULT_TABLE table=test-table [id=eb73c53b5e93460ea7ed29390c6408ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:08.617700 11266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.605393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41074 (local address 127.8.208.66:40093)
0929 02:58:08.607139 (+ 1746us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:08.607151 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:08.608001 (+ 850us) server_negotiation.cc:407] Connection header received
0929 02:58:08.608708 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:08.608717 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:08.608978 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:08.609145 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:08.610453 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.612020 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.615026 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.616366 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.616522 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.616712 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.617240 (+ 528us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.617426 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1569,"thread_start_us":1430,"threads_started":1}
I20260929 02:58:08.622426 11181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1b12a8695d4414a18688da5b354a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:08.622254 10903 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1b12a8695d4414a18688da5b354a92 (DEFAULT_TABLE table=test-table [id=eb73c53b5e93460ea7ed29390c6408ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:08.623632 10903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1b12a8695d4414a18688da5b354a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:08.624665 11042 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1b12a8695d4414a18688da5b354a92 (DEFAULT_TABLE table=test-table [id=eb73c53b5e93460ea7ed29390c6408ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:08.625967 11042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1b12a8695d4414a18688da5b354a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:08.633755 11270 tablet_bootstrap.cc:492] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b: Bootstrap starting.
I20260929 02:58:08.637331 11270 tablet_bootstrap.cc:654] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:08.637852 11271 tablet_bootstrap.cc:492] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb: Bootstrap starting.
I20260929 02:58:08.638700 11270 log.cc:824] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:08.641247 11271 tablet_bootstrap.cc:654] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:08.641856 11270 tablet_bootstrap.cc:492] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b: No bootstrap required, opened a new log
I20260929 02:58:08.642132 11270 ts_tablet_manager.cc:1397] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 02:58:08.642483 11271 log.cc:824] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:08.645828 11273 tablet_bootstrap.cc:492] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a: Bootstrap starting.
I20260929 02:58:08.648623 11273 tablet_bootstrap.cc:654] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:08.649761 11273 log.cc:824] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:08.651872 11271 tablet_bootstrap.cc:492] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb: No bootstrap required, opened a new log
I20260929 02:58:08.652163 11273 tablet_bootstrap.cc:492] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a: No bootstrap required, opened a new log
I20260929 02:58:08.652148 11271 ts_tablet_manager.cc:1397] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260929 02:58:08.652397 11273 ts_tablet_manager.cc:1397] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 02:58:08.654234 11270 raft_consensus.cc:348] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } }
I20260929 02:58:08.654826 11270 raft_consensus.cc:374] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:08.654989 11270 raft_consensus.cc:729] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802a4d73449747d69f6c3cacbf95886b, State: Initialized, Role: FOLLOWER
I20260929 02:58:08.655619 11270 consensus_queue.cc:260] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } }
I20260929 02:58:08.656737 11251 heartbeater.cc:503] Master 127.8.208.126:33131 was elected leader, sending a full tablet report...
I20260929 02:58:08.657418 11270 ts_tablet_manager.cc:1428] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260929 02:58:08.661489 11273 raft_consensus.cc:348] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } }
I20260929 02:58:08.661911 11273 raft_consensus.cc:374] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:08.662065 11273 raft_consensus.cc:729] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93235c192b3d43f0923782e8288ce61a, State: Initialized, Role: FOLLOWER
I20260929 02:58:08.662307 11271 raft_consensus.cc:348] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } }
I20260929 02:58:08.662716 11271 raft_consensus.cc:374] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:08.662829 11271 raft_consensus.cc:729] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dd9a6f086842cbb07f628dc0b94feb, State: Initialized, Role: FOLLOWER
I20260929 02:58:08.662752 11273 consensus_queue.cc:260] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } }
I20260929 02:58:08.663419 11271 consensus_queue.cc:260] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } }
I20260929 02:58:08.663924 11112 heartbeater.cc:503] Master 127.8.208.126:33131 was elected leader, sending a full tablet report...
I20260929 02:58:08.664609 11273 ts_tablet_manager.cc:1428] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260929 02:58:08.665009 11271 ts_tablet_manager.cc:1428] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb: Time spent starting tablet: real 0.013s user 0.011s sys 0.002s
I20260929 02:58:08.748066 11278 raft_consensus.cc:482] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:08.748433 11278 raft_consensus.cc:504] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } }
I20260929 02:58:08.750180 11278 leader_election.cc:288] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 802a4d73449747d69f6c3cacbf95886b (127.8.208.67:34935), 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093)
I20260929 02:58:08.757486 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.750857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:39055 (local address 127.8.208.67:34935)
0929 02:58:08.751313 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:08.751321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:08.751905 (+ 584us) server_negotiation.cc:407] Connection header received
0929 02:58:08.752068 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:08.752073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:08.752147 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:08.752269 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:08.752974 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.753779 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.755721 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.756471 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.756656 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.756729 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.757059 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.757207 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311}
I20260929 02:58:08.757499 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.750718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:34935 (local address 127.8.208.65:39055)
0929 02:58:08.751784 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:08.751802 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:08.751935 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:08.752326 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:08.752334 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:08.752355 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:08.752786 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.752802 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.753963 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.753968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.755566 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.755584 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.756669 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.756675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.756862 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.756909 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 02:58:08.757005 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 02:58:08.757133 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"thread_start_us":685,"threads_started":1}
I20260929 02:58:08.757655 11266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.750759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:51385 (local address 127.8.208.66:40093)
0929 02:58:08.751138 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:08.751145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:08.751621 (+ 476us) server_negotiation.cc:407] Connection header received
0929 02:58:08.751820 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:08.751828 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:08.751917 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:08.752018 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:08.752619 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.753684 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.755641 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.756573 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.756768 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.756850 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.757246 (+ 396us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.757407 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
I20260929 02:58:08.758209 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.750594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:40093 (local address 127.8.208.65:51385)
0929 02:58:08.751464 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:08.751490 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:08.751676 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:08.752049 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:08.752054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:08.752078 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:08.752471 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.752482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.753812 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.753819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.755472 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.755484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.756886 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.756892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.757048 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.757077 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 02:58:08.757834 (+ 757us) client_negotiation.cc:240] Negotiation successful
0929 02:58:08.757946 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":401,"threads_started":1}
I20260929 02:58:08.758939 11205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1b12a8695d4414a18688da5b354a92" candidate_uuid: "35dd9a6f086842cbb07f628dc0b94feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "802a4d73449747d69f6c3cacbf95886b" is_pre_election: true
I20260929 02:58:08.759262 11065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1b12a8695d4414a18688da5b354a92" candidate_uuid: "35dd9a6f086842cbb07f628dc0b94feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93235c192b3d43f0923782e8288ce61a" is_pre_election: true
I20260929 02:58:08.759541 11205 raft_consensus.cc:2454] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35dd9a6f086842cbb07f628dc0b94feb in term 0.
I20260929 02:58:08.759788 11065 raft_consensus.cc:2454] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35dd9a6f086842cbb07f628dc0b94feb in term 0.
I20260929 02:58:08.760272 10856 leader_election.cc:302] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dd9a6f086842cbb07f628dc0b94feb, 802a4d73449747d69f6c3cacbf95886b; no voters:
I20260929 02:58:08.760661 11278 raft_consensus.cc:2781] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:08.760808 11278 raft_consensus.cc:482] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:08.760911 11278 raft_consensus.cc:3037] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:08.763068 11278 raft_consensus.cc:504] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } }
I20260929 02:58:08.763669 11278 leader_election.cc:288] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [CANDIDATE]: Term 1 election: Requested vote from peers 802a4d73449747d69f6c3cacbf95886b (127.8.208.67:34935), 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093)
I20260929 02:58:08.764025 11205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1b12a8695d4414a18688da5b354a92" candidate_uuid: "35dd9a6f086842cbb07f628dc0b94feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "802a4d73449747d69f6c3cacbf95886b"
I20260929 02:58:08.764154 11065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1b12a8695d4414a18688da5b354a92" candidate_uuid: "35dd9a6f086842cbb07f628dc0b94feb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93235c192b3d43f0923782e8288ce61a"
I20260929 02:58:08.764240 11205 raft_consensus.cc:3037] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:08.764357 11065 raft_consensus.cc:3037] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Advancing to term 1
W20260929 02:58:08.765691 11113 tablet.cc:2369] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:58:08.767642 11205 raft_consensus.cc:2454] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35dd9a6f086842cbb07f628dc0b94feb in term 1.
I20260929 02:58:08.767642 11065 raft_consensus.cc:2454] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35dd9a6f086842cbb07f628dc0b94feb in term 1.
I20260929 02:58:08.768079 10853 leader_election.cc:302] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dd9a6f086842cbb07f628dc0b94feb, 93235c192b3d43f0923782e8288ce61a; no voters:
I20260929 02:58:08.768379 11278 raft_consensus.cc:2781] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:08.768852 11278 raft_consensus.cc:686] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 1 LEADER]: Becoming Leader. State: Replica: 35dd9a6f086842cbb07f628dc0b94feb, State: Running, Role: LEADER
I20260929 02:58:08.769317 11278 consensus_queue.cc:237] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } }
W20260929 02:58:08.773900 10974 tablet.cc:2369] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:58:08.773995 10765 catalog_manager.cc:5495] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb reported cstate change: term changed from 0 to 1, leader changed from <none> to 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65). New cstate: current_term: 1 leader_uuid: "35dd9a6f086842cbb07f628dc0b94feb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } health_report { overall_health: HEALTHY } } }
W20260929 02:58:08.778469 11252 tablet.cc:2369] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 02:58:08 dist-test-slave-m1rv krb5kdc[10721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650686, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:08.827435 11288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.815342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33131 (local address 127.0.0.1:60080)
0929 02:58:08.815897 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:08.815913 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:08.816038 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:08.816363 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:08.816366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:08.817004 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:08.817316 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.817326 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.818307 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.818310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.819273 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.819283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.819964 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.819968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.820085 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.820673 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:08.820691 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:08.822843 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:08.825360 (+ 2517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:08.825369 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:08.825375 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:08.825772 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:08.826072 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:08.826074 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:08.826077 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:08.826205 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:08.826655 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:08.826661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:08.826799 (+ 138us) client_negotiation.cc:769] Sending connection context
0929 02:58:08.826929 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 02:58:08.827124 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":287,"threads_started":1}
I20260929 02:58:08.827467 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.815264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60080 (local address 127.8.208.126:33131)
0929 02:58:08.815432 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:08.815436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:08.815997 (+ 561us) server_negotiation.cc:407] Connection header received
0929 02:58:08.816131 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:08.816134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:08.816215 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:08.816316 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:08.817458 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.818205 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.819386 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.819811 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.819900 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.822996 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:08.823001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:08.823004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:08.823030 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:08.825241 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:08.825864 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:08.825867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:08.825869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:08.825987 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:08.826286 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:08.826289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:08.826291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:08.826551 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:08.826651 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.827020 (+ 369us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.827231 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":64}
I20260929 02:58:08.832211 10765 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:60080:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 02:58:08.833321 10765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:08.839481 10903 tablet_service.cc:1467] Processing CreateTablet for tablet 7a2d13674def43078e84a1ff2a6a1913 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e5f946d84f614ea3a46755db12570238]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:58:08.839654 11042 tablet_service.cc:1467] Processing CreateTablet for tablet 7a2d13674def43078e84a1ff2a6a1913 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e5f946d84f614ea3a46755db12570238]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:58:08.839772 11181 tablet_service.cc:1467] Processing CreateTablet for tablet 7a2d13674def43078e84a1ff2a6a1913 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e5f946d84f614ea3a46755db12570238]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:58:08.840499 10903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a2d13674def43078e84a1ff2a6a1913. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:08.840530 11042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a2d13674def43078e84a1ff2a6a1913. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:08.840530 11181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a2d13674def43078e84a1ff2a6a1913. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:08.846374 11271 tablet_bootstrap.cc:492] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb: Bootstrap starting.
I20260929 02:58:08.847543 11273 tablet_bootstrap.cc:492] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: Bootstrap starting.
I20260929 02:58:08.847579 11270 tablet_bootstrap.cc:492] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b: Bootstrap starting.
I20260929 02:58:08.847874 11271 tablet_bootstrap.cc:654] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:08.849282 11270 tablet_bootstrap.cc:654] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:08.849373 11273 tablet_bootstrap.cc:654] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:08.850234 11271 tablet_bootstrap.cc:492] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb: No bootstrap required, opened a new log
I20260929 02:58:08.850425 11271 ts_tablet_manager.cc:1397] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 02:58:08.851248 11273 tablet_bootstrap.cc:492] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: No bootstrap required, opened a new log
I20260929 02:58:08.851336 11273 ts_tablet_manager.cc:1397] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260929 02:58:08.851430 11271 raft_consensus.cc:348] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:08.851732 11271 raft_consensus.cc:374] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:08.851835 11271 raft_consensus.cc:729] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dd9a6f086842cbb07f628dc0b94feb, State: Initialized, Role: FOLLOWER
I20260929 02:58:08.851892 11273 raft_consensus.cc:348] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:08.852182 11273 raft_consensus.cc:374] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:08.852334 11273 raft_consensus.cc:729] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93235c192b3d43f0923782e8288ce61a, State: Initialized, Role: FOLLOWER
I20260929 02:58:08.852324 11271 consensus_queue.cc:260] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:08.852680 11273 consensus_queue.cc:260] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:08.852906 11278 tablet_replica.cc:439] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } }
I20260929 02:58:08.853258 11271 ts_tablet_manager.cc:1428] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260929 02:58:08.853161 11277 tablet_replica.cc:439] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } }
I20260929 02:58:08.853559 11277 tablet_replica.cc:442] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:08.853775 11278 tablet_replica.cc:442] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:08.854640 11273 ts_tablet_manager.cc:1428] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260929 02:58:08.862468 11270 tablet_bootstrap.cc:492] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b: No bootstrap required, opened a new log
I20260929 02:58:08.862620 11270 ts_tablet_manager.cc:1397] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.012s
I20260929 02:58:08.863355 11270 raft_consensus.cc:348] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:08.863615 11270 raft_consensus.cc:374] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:08.863688 11270 raft_consensus.cc:729] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802a4d73449747d69f6c3cacbf95886b, State: Initialized, Role: FOLLOWER
I20260929 02:58:08.863857 11270 consensus_queue.cc:260] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:08.864434 11270 ts_tablet_manager.cc:1428] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260929 02:58:08.864275 11276 tablet_replica.cc:439] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } }
I20260929 02:58:08.864703 11276 tablet_replica.cc:442] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:08.898939 11277 raft_consensus.cc:482] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:08.899255 11277 raft_consensus.cc:504] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:08.901010 11277 leader_election.cc:288] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035), 802a4d73449747d69f6c3cacbf95886b (127.8.208.67:34935)
I20260929 02:58:08.908541 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.901528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:46701 (local address 127.8.208.67:34935)
0929 02:58:08.901888 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:08.901894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:08.902505 (+ 611us) server_negotiation.cc:407] Connection header received
0929 02:58:08.902753 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:08.902756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:08.902813 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:08.902884 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:08.903702 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.904845 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.906753 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.907538 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.907697 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.907774 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.908109 (+ 335us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.908239 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224}
I20260929 02:58:08.908671 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.901289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:34935 (local address 127.8.208.66:46701)
0929 02:58:08.902406 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:08.902431 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:08.902601 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:08.902916 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:08.902922 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:08.902961 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:08.903507 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.903527 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.905004 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.905008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.906604 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.906615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.907745 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.907751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.907925 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.907965 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 02:58:08.908074 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 02:58:08.908226 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":231,"spinlock_wait_cycles":1280,"thread_start_us":568,"threads_started":1}
I20260929 02:58:08.909351 11205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2d13674def43078e84a1ff2a6a1913" candidate_uuid: "93235c192b3d43f0923782e8288ce61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "802a4d73449747d69f6c3cacbf95886b" is_pre_election: true
I20260929 02:58:08.909621 11205 raft_consensus.cc:2454] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93235c192b3d43f0923782e8288ce61a in term 0.
I20260929 02:58:08.909775 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.901640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:41151 (local address 127.8.208.65:41035)
0929 02:58:08.901898 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:08.901904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:08.902234 (+ 330us) server_negotiation.cc:407] Connection header received
0929 02:58:08.902531 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:08.902539 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:08.902624 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:08.902696 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:08.903720 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.904793 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.907609 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.908667 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.908818 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.908909 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.909401 (+ 492us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.909533 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 02:58:08.909895 11292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.901289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:41035 (local address 127.8.208.66:41151)
0929 02:58:08.902113 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:08.902140 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:08.902325 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:08.902916 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:08.902922 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:08.902968 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:08.903538 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.903550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.904944 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.904950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.907440 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.907456 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.908980 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.908987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.909166 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.909240 (+ 74us) client_negotiation.cc:769] Sending connection context
0929 02:58:08.909353 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 02:58:08.909490 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":27,"spinlock_wait_cycles":5248,"thread_start_us":590,"threads_started":1}
I20260929 02:58:08.910177 10996 leader_election.cc:302] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 802a4d73449747d69f6c3cacbf95886b, 93235c192b3d43f0923782e8288ce61a; no voters:
I20260929 02:58:08.910590 11277 raft_consensus.cc:2781] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:08.910722 11277 raft_consensus.cc:482] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:08.910827 11277 raft_consensus.cc:3037] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:08.910836 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2d13674def43078e84a1ff2a6a1913" candidate_uuid: "93235c192b3d43f0923782e8288ce61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dd9a6f086842cbb07f628dc0b94feb" is_pre_election: true
I20260929 02:58:08.911284 10923 raft_consensus.cc:2454] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93235c192b3d43f0923782e8288ce61a in term 0.
I20260929 02:58:08.914244 11277 raft_consensus.cc:504] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:08.914912 11277 leader_election.cc:288] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [CANDIDATE]: Term 1 election: Requested vote from peers 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035), 802a4d73449747d69f6c3cacbf95886b (127.8.208.67:34935)
I20260929 02:58:08.915320 11205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2d13674def43078e84a1ff2a6a1913" candidate_uuid: "93235c192b3d43f0923782e8288ce61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "802a4d73449747d69f6c3cacbf95886b"
I20260929 02:58:08.915345 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2d13674def43078e84a1ff2a6a1913" candidate_uuid: "93235c192b3d43f0923782e8288ce61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dd9a6f086842cbb07f628dc0b94feb"
I20260929 02:58:08.915534 11205 raft_consensus.cc:3037] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:08.915544 10923 raft_consensus.cc:3037] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:08.917914 11205 raft_consensus.cc:2454] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93235c192b3d43f0923782e8288ce61a in term 1.
I20260929 02:58:08.918354 10996 leader_election.cc:302] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 802a4d73449747d69f6c3cacbf95886b, 93235c192b3d43f0923782e8288ce61a; no voters:
I20260929 02:58:08.918694 11277 raft_consensus.cc:2781] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:08.918964 10923 raft_consensus.cc:2454] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93235c192b3d43f0923782e8288ce61a in term 1.
I20260929 02:58:08.919377 11277 raft_consensus.cc:686] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [term 1 LEADER]: Becoming Leader. State: Replica: 93235c192b3d43f0923782e8288ce61a, State: Running, Role: LEADER
I20260929 02:58:08.919910 11277 consensus_queue.cc:237] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:08.923440 11294 tablet_replica.cc:439] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: TxnStatusTablet state changed. Reason: New leader 93235c192b3d43f0923782e8288ce61a. Latest consensus state: current_term: 1 leader_uuid: "93235c192b3d43f0923782e8288ce61a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } }
I20260929 02:58:08.923795 11294 tablet_replica.cc:442] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:58:08.924469 11296 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 02:58:08.924656 11296 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 02:58:08.925545 10765 catalog_manager.cc:5495] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a reported cstate change: term changed from 0 to 1, leader changed from <none> to 93235c192b3d43f0923782e8288ce61a (127.8.208.66). New cstate: current_term: 1 leader_uuid: "93235c192b3d43f0923782e8288ce61a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } health_report { overall_health: UNKNOWN } } }
I20260929 02:58:08.957307 11288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.950796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:40093 (local address 127.0.0.1:41086)
0929 02:58:08.951081 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:08.951097 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:08.951293 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:08.951776 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:08.951782 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:08.951794 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:58:08.952002 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.952013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.953200 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.953208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.954472 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:08.954485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.955461 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.955469 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:08.955656 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.955714 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:58:08.956636 (+ 922us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:58:08.956662 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 02:58:08.956777 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 02:58:08.956941 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260929 02:58:08.957301 11266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:08.950781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41086 (local address 127.8.208.66:40093)
0929 02:58:08.950962 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:08.950969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:08.951173 (+ 204us) server_negotiation.cc:407] Connection header received
0929 02:58:08.951514 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:08.951518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:08.951603 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:08.951754 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:08.952139 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.953039 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.954621 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:08.955287 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:08.955423 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:08.955912 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:08.956510 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:08.956674 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:08.956806 (+ 132us) server_negotiation.cc:299] Negotiation successful
0929 02:58:08.956890 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:58:08.965523 11205 raft_consensus.cc:1261] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Refusing update from remote peer 93235c192b3d43f0923782e8288ce61a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:08.965480 10923 raft_consensus.cc:1261] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Refusing update from remote peer 93235c192b3d43f0923782e8288ce61a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:08.966521 11294 consensus_queue.cc:1035] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [LEADER]: Connected to new peer: Peer: permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:08.966885 11277 consensus_queue.cc:1035] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:08.977419 11276 tablet_replica.cc:439] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b: TxnStatusTablet state changed. Reason: New leader 93235c192b3d43f0923782e8288ce61a. Latest consensus state: current_term: 1 leader_uuid: "93235c192b3d43f0923782e8288ce61a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } }
I20260929 02:58:08.977875 11276 tablet_replica.cc:442] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:08.979192 11278 tablet_replica.cc:439] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb: TxnStatusTablet state changed. Reason: New leader 93235c192b3d43f0923782e8288ce61a. Latest consensus state: current_term: 1 leader_uuid: "93235c192b3d43f0923782e8288ce61a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } }
I20260929 02:58:08.979557 11278 tablet_replica.cc:442] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:08.987259 11299 mvcc.cc:205] Tried to move back new op lower bound from 7334505221992304640 to 7334505221819731968. Current Snapshot: MvccSnapshot[applied={T|T < 7334505221992304640}]
I20260929 02:58:08.989511 11277 tablet_replica.cc:439] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93235c192b3d43f0923782e8288ce61a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } }
I20260929 02:58:08.989945 11277 tablet_replica.cc:442] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:58:08.994508 11278 tablet_replica.cc:439] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93235c192b3d43f0923782e8288ce61a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } }
I20260929 02:58:08.994925 11278 tablet_replica.cc:442] T 7a2d13674def43078e84a1ff2a6a1913 P 35dd9a6f086842cbb07f628dc0b94feb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:08.996937 11294 tablet_replica.cc:439] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93235c192b3d43f0923782e8288ce61a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } }
I20260929 02:58:08.997264 11294 tablet_replica.cc:442] T 7a2d13674def43078e84a1ff2a6a1913 P 93235c192b3d43f0923782e8288ce61a: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:58:08.998927 11300 mvcc.cc:205] Tried to move back new op lower bound from 7334505221992304640 to 7334505221819731968. Current Snapshot: MvccSnapshot[applied={T|T < 7334505221992304640}]
I20260929 02:58:09.003384 11276 tablet_replica.cc:439] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93235c192b3d43f0923782e8288ce61a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } } }
I20260929 02:58:09.003623 11276 tablet_replica.cc:442] T 7a2d13674def43078e84a1ff2a6a1913 P 802a4d73449747d69f6c3cacbf95886b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:09.011554 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.005079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56870 (local address 127.8.208.65:41035)
0929 02:58:09.005298 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:09.005304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:09.005800 (+ 496us) server_negotiation.cc:407] Connection header received
0929 02:58:09.006049 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:09.006054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:09.006193 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:09.006373 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:09.006878 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.007703 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.009202 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.009674 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.009846 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.010305 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:09.010876 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:09.011049 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:09.011111 (+ 62us) server_negotiation.cc:299] Negotiation successful
0929 02:58:09.011199 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:58:09.031165 11266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.023387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:36501 (local address 127.8.208.66:40093)
0929 02:58:09.023608 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:09.023617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:09.024259 (+ 642us) server_negotiation.cc:407] Connection header received
0929 02:58:09.024464 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:09.024470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:09.024553 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:09.024640 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:09.025394 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.026722 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.028754 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.029607 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.029749 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.030419 (+ 670us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:09.030546 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:09.030638 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:09.030803 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 02:58:09.030884 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:58:09.031484 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.023358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:40093 (local address 127.8.208.65:36501)
0929 02:58:09.024132 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:09.024154 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:09.024331 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:09.024802 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:09.024808 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:09.024821 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:58:09.025238 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:09.025251 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.026867 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.026872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:09.028598 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:09.028611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.029915 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.029923 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:09.030113 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.030186 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:58:09.030660 (+ 474us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:58:09.030675 (+ 15us) client_negotiation.cc:769] Sending connection context
0929 02:58:09.030770 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 02:58:09.030924 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":370,"threads_started":1}
I20260929 02:58:09.043255 11205 raft_consensus.cc:1261] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Refusing update from remote peer 35dd9a6f086842cbb07f628dc0b94feb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:09.043486 11065 raft_consensus.cc:1261] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Refusing update from remote peer 35dd9a6f086842cbb07f628dc0b94feb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:09.044350 11278 consensus_queue.cc:1035] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:09.044759 11281 consensus_queue.cc:1035] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [LEADER]: Connected to new peer: Peer: permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:09.051134 11301 mvcc.cc:205] Tried to move back new op lower bound from 7334505222313693184 to 7334505221202862080. Current Snapshot: MvccSnapshot[applied={T|T < 7334505222313693184}]
I20260929 02:58:09.082223 11315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.075797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:41035 (local address 127.8.208.66:41619)
0929 02:58:09.076425 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:09.076447 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:09.076591 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:09.077062 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:09.077066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:09.077075 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:58:09.077460 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:09.077469 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.078474 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.078482 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:09.080192 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:09.080204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.080951 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.080959 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:09.081138 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.081246 (+ 108us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:58:09.081636 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:58:09.081658 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:58:09.081753 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 02:58:09.081908 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":310,"threads_started":1}
I20260929 02:58:09.082242 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.075892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:41619 (local address 127.8.208.65:41035)
0929 02:58:09.076075 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:09.076083 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:09.076519 (+ 436us) server_negotiation.cc:407] Connection header received
0929 02:58:09.076813 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:09.076819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:09.076923 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:09.077024 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:09.077617 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.078337 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.080332 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.080759 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.080897 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.081433 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:09.081531 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:09.081637 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:09.081807 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 02:58:09.081901 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:58:09.126150 10766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:09.126484 10766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:09.130918 10766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4: Sending DeleteTablet for 3 replicas of tablet 7e1b12a8695d4414a18688da5b354a92
I20260929 02:58:09.132237 11181 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1b12a8695d4414a18688da5b354a92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:09 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:35306
I20260929 02:58:09.132249 10903 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1b12a8695d4414a18688da5b354a92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:09 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:56856
I20260929 02:58:09.132473 11042 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1b12a8695d4414a18688da5b354a92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:09 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:41074
I20260929 02:58:09.135035 10766 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:58:09.135960 10766 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:09.145727 11321 tablet_replica.cc:330] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a: stopping tablet replica
I20260929 02:58:09.146126 11320 tablet_replica.cc:330] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b: stopping tablet replica
I20260929 02:58:09.145788 11319 tablet_replica.cc:330] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb: stopping tablet replica
I20260929 02:58:09.146996 11319 raft_consensus.cc:2229] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:09.148514 11321 raft_consensus.cc:2229] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:09.148517 11041 tablet_service.cc:1467] Processing CreateTablet for tablet 84ea3b53e33d4939acf7919220c727ce (DEFAULT_TABLE table=test-table [id=aeffc93e2e8c4a56af84dd56e3cd8da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:09.148947 11321 raft_consensus.cc:2258] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:09.149490 11041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84ea3b53e33d4939acf7919220c727ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:09.150394 11181 tablet_service.cc:1467] Processing CreateTablet for tablet 84ea3b53e33d4939acf7919220c727ce (DEFAULT_TABLE table=test-table [id=aeffc93e2e8c4a56af84dd56e3cd8da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:09.150878 11181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84ea3b53e33d4939acf7919220c727ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:09.156634 11320 raft_consensus.cc:2229] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:09.157312 11273 tablet_bootstrap.cc:492] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a: Bootstrap starting.
I20260929 02:58:09.148088 10903 tablet_service.cc:1467] Processing CreateTablet for tablet 84ea3b53e33d4939acf7919220c727ce (DEFAULT_TABLE table=test-table [id=aeffc93e2e8c4a56af84dd56e3cd8da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:09.157755 11320 raft_consensus.cc:2258] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:09.157796 10903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84ea3b53e33d4939acf7919220c727ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:09.159214 11273 tablet_bootstrap.cc:654] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:09.159752 11319 raft_consensus.cc:2258] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:09.164351 11270 tablet_bootstrap.cc:492] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b: Bootstrap starting.
I20260929 02:58:09.164593 11319 ts_tablet_manager.cc:1905] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:09.166078 11270 tablet_bootstrap.cc:654] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:09.166688 11320 ts_tablet_manager.cc:1905] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:09.173358 11271 tablet_bootstrap.cc:492] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb: Bootstrap starting.
I20260929 02:58:09.174031 11321 ts_tablet_manager.cc:1905] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:09.174937 11270 tablet_bootstrap.cc:492] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b: No bootstrap required, opened a new log
I20260929 02:58:09.175040 11271 tablet_bootstrap.cc:654] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:09.175179 11270 ts_tablet_manager.cc:1397] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260929 02:58:09.176064 11270 raft_consensus.cc:348] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:09.176431 11270 raft_consensus.cc:374] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:09.176563 11270 raft_consensus.cc:729] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802a4d73449747d69f6c3cacbf95886b, State: Initialized, Role: FOLLOWER
I20260929 02:58:09.176929 11270 consensus_queue.cc:260] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:09.180262 11270 ts_tablet_manager.cc:1428] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260929 02:58:09.181200 11319 ts_tablet_manager.cc:1918] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:09.181371 11319 log.cc:1196] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/7e1b12a8695d4414a18688da5b354a92
I20260929 02:58:09.181919 11319 ts_tablet_manager.cc:1939] T 7e1b12a8695d4414a18688da5b354a92 P 35dd9a6f086842cbb07f628dc0b94feb: Deleting consensus metadata
I20260929 02:58:09.183797 10752 catalog_manager.cc:4841] TS 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035): tablet 7e1b12a8695d4414a18688da5b354a92 (table test-table [id=eb73c53b5e93460ea7ed29390c6408ce]) successfully deleted
I20260929 02:58:09.187067 11321 ts_tablet_manager.cc:1918] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:09.187260 11273 tablet_bootstrap.cc:492] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a: No bootstrap required, opened a new log
I20260929 02:58:09.187291 11321 log.cc:1196] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/7e1b12a8695d4414a18688da5b354a92
I20260929 02:58:09.187554 11273 ts_tablet_manager.cc:1397] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.005s
I20260929 02:58:09.187968 11321 ts_tablet_manager.cc:1939] T 7e1b12a8695d4414a18688da5b354a92 P 93235c192b3d43f0923782e8288ce61a: Deleting consensus metadata
I20260929 02:58:09.188535 11273 raft_consensus.cc:348] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:09.188899 11273 raft_consensus.cc:374] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:09.189023 11273 raft_consensus.cc:729] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93235c192b3d43f0923782e8288ce61a, State: Initialized, Role: FOLLOWER
I20260929 02:58:09.189498 11271 tablet_bootstrap.cc:492] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb: No bootstrap required, opened a new log
I20260929 02:58:09.189396 11273 consensus_queue.cc:260] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:09.189721 11271 ts_tablet_manager.cc:1397] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.008s
I20260929 02:58:09.190660 10750 catalog_manager.cc:4841] TS 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093): tablet 7e1b12a8695d4414a18688da5b354a92 (table test-table [id=eb73c53b5e93460ea7ed29390c6408ce]) successfully deleted
I20260929 02:58:09.191242 11320 ts_tablet_manager.cc:1918] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:09.191277 11273 ts_tablet_manager.cc:1428] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260929 02:58:09.191433 11320 log.cc:1196] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/7e1b12a8695d4414a18688da5b354a92
I20260929 02:58:09.192055 11320 ts_tablet_manager.cc:1939] T 7e1b12a8695d4414a18688da5b354a92 P 802a4d73449747d69f6c3cacbf95886b: Deleting consensus metadata
I20260929 02:58:09.191936 11271 raft_consensus.cc:348] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:09.192181 11271 raft_consensus.cc:374] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:09.192276 11271 raft_consensus.cc:729] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dd9a6f086842cbb07f628dc0b94feb, State: Initialized, Role: FOLLOWER
I20260929 02:58:09.192593 11271 consensus_queue.cc:260] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:09.194594 10753 catalog_manager.cc:4841] TS 802a4d73449747d69f6c3cacbf95886b (127.8.208.67:34935): tablet 7e1b12a8695d4414a18688da5b354a92 (table test-table [id=eb73c53b5e93460ea7ed29390c6408ce]) successfully deleted
I20260929 02:58:09.196111 11271 ts_tablet_manager.cc:1428] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260929 02:58:09.229321 11276 raft_consensus.cc:482] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:09.229602 11276 raft_consensus.cc:504] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:09.231387 11276 leader_election.cc:288] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035), 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093)
I20260929 02:58:09.251453 11266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.232391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:42339 (local address 127.8.208.66:40093)
0929 02:58:09.232566 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:09.232572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:09.243320 (+ 10748us) server_negotiation.cc:407] Connection header received
0929 02:58:09.243620 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:09.243626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:09.243706 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:09.243768 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:09.244576 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.245950 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.248703 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.250118 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.250284 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.250361 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:09.250992 (+ 631us) server_negotiation.cc:299] Negotiation successful
0929 02:58:09.251137 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:58:09.252297 11325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.231913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:40093 (local address 127.8.208.67:42339)
0929 02:58:09.243211 (+ 11298us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:09.243230 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:09.243443 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:09.244017 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:09.244022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:09.244051 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:09.244367 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:09.244380 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.246182 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.246187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:09.248524 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:09.248539 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.250613 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.250620 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:09.250796 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.250831 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 02:58:09.251737 (+ 906us) client_negotiation.cc:240] Negotiation successful
0929 02:58:09.251882 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11159,"spinlock_wait_cycles":37632,"thread_start_us":10654,"threads_started":1}
I20260929 02:58:09.252964 11063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ea3b53e33d4939acf7919220c727ce" candidate_uuid: "802a4d73449747d69f6c3cacbf95886b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93235c192b3d43f0923782e8288ce61a" is_pre_election: true
I20260929 02:58:09.253252 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.232000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:56107 (local address 127.8.208.65:41035)
0929 02:58:09.232194 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:09.232214 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:09.243024 (+ 10810us) server_negotiation.cc:407] Connection header received
0929 02:58:09.243659 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:09.243666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:09.243752 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:09.243835 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:09.245048 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.246463 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.249427 (+ 2964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.250999 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.251163 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.251270 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:09.252668 (+ 1398us) server_negotiation.cc:299] Negotiation successful
0929 02:58:09.252894 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:58:09.253298 11063 raft_consensus.cc:2454] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 802a4d73449747d69f6c3cacbf95886b in term 0.
I20260929 02:58:09.253875 11132 leader_election.cc:302] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 802a4d73449747d69f6c3cacbf95886b, 93235c192b3d43f0923782e8288ce61a; no voters:
I20260929 02:58:09.254705 11276 raft_consensus.cc:2781] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:09.254845 11276 raft_consensus.cc:482] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:09.254949 11276 raft_consensus.cc:3037] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:09.255149 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.231619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:41035 (local address 127.8.208.67:56107)
0929 02:58:09.242885 (+ 11266us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:09.242914 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:09.243409 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:09.244102 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:09.244112 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:09.244235 (+ 123us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:09.244809 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:09.244823 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.246765 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.246771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:09.249220 (+ 2449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:09.249238 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.251480 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.251487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:09.252108 (+ 621us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.252142 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:58:09.254237 (+ 2095us) client_negotiation.cc:240] Negotiation successful
0929 02:58:09.254406 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11106,"spinlock_wait_cycles":896,"thread_start_us":11429,"threads_started":1}
I20260929 02:58:09.255998 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ea3b53e33d4939acf7919220c727ce" candidate_uuid: "802a4d73449747d69f6c3cacbf95886b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dd9a6f086842cbb07f628dc0b94feb" is_pre_election: true
I20260929 02:58:09.256379 10923 raft_consensus.cc:2454] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 802a4d73449747d69f6c3cacbf95886b in term 0.
I20260929 02:58:09.258348 11276 raft_consensus.cc:504] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:09.259405 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ea3b53e33d4939acf7919220c727ce" candidate_uuid: "802a4d73449747d69f6c3cacbf95886b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dd9a6f086842cbb07f628dc0b94feb"
I20260929 02:58:09.259610 10923 raft_consensus.cc:3037] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:09.260311 11276 leader_election.cc:288] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [CANDIDATE]: Term 1 election: Requested vote from peers 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035), 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093)
I20260929 02:58:09.261142 11063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ea3b53e33d4939acf7919220c727ce" candidate_uuid: "802a4d73449747d69f6c3cacbf95886b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93235c192b3d43f0923782e8288ce61a"
I20260929 02:58:09.261422 11063 raft_consensus.cc:3037] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:09.262725 10923 raft_consensus.cc:2454] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 802a4d73449747d69f6c3cacbf95886b in term 1.
I20260929 02:58:09.263116 11134 leader_election.cc:302] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dd9a6f086842cbb07f628dc0b94feb, 802a4d73449747d69f6c3cacbf95886b; no voters:
I20260929 02:58:09.263439 11276 raft_consensus.cc:2781] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:09.264726 11063 raft_consensus.cc:2454] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 802a4d73449747d69f6c3cacbf95886b in term 1.
I20260929 02:58:09.265717 11276 raft_consensus.cc:686] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 1 LEADER]: Becoming Leader. State: Replica: 802a4d73449747d69f6c3cacbf95886b, State: Running, Role: LEADER
I20260929 02:58:09.266175 11276 consensus_queue.cc:237] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:09.270212 10765 catalog_manager.cc:5495] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b reported cstate change: term changed from 0 to 1, leader changed from <none> to 802a4d73449747d69f6c3cacbf95886b (127.8.208.67). New cstate: current_term: 1 leader_uuid: "802a4d73449747d69f6c3cacbf95886b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } health_report { overall_health: HEALTHY } } }
I20260929 02:58:09.308895 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.301893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35316 (local address 127.8.208.67:34935)
0929 02:58:09.302071 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:09.302078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:09.302099 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:58:09.302220 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:09.302227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:09.302352 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:09.302538 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:09.302965 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.304084 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.306060 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.306779 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.306931 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.307445 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:09.308204 (+ 759us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:09.308378 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:09.308450 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 02:58:09.308539 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:58:09.328538 11331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.321008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:40093 (local address 127.8.208.67:48249)
0929 02:58:09.321681 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:09.321701 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:09.321829 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:09.322342 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:09.322347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:09.322360 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:58:09.322728 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:09.322741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.324264 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.324270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:09.326033 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:09.326048 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.327144 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.327150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:09.327333 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.327402 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:58:09.327908 (+ 506us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:58:09.327919 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 02:58:09.328002 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 02:58:09.328144 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":795,"threads_started":1}
I20260929 02:58:09.328737 11266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.321550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:48249 (local address 127.8.208.66:40093)
0929 02:58:09.321755 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:09.321762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:09.321787 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:58:09.322007 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:09.322013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:09.322095 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:09.322157 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:09.322910 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.324112 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.326199 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.326928 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.327068 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.327621 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:09.327793 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:09.327906 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:09.328052 (+ 146us) server_negotiation.cc:299] Negotiation successful
0929 02:58:09.328407 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 02:58:09.340209 10923 raft_consensus.cc:1261] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Refusing update from remote peer 802a4d73449747d69f6c3cacbf95886b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:09.340273 11063 raft_consensus.cc:1261] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Refusing update from remote peer 802a4d73449747d69f6c3cacbf95886b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:09.341034 11327 consensus_queue.cc:1035] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:09.341524 11276 consensus_queue.cc:1035] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [LEADER]: Connected to new peer: Peer: permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:09.400520 11315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.378234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:34935 (local address 127.8.208.66:54315)
0929 02:58:09.378672 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:09.378695 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:09.378823 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:09.379762 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:09.379773 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:09.379789 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:58:09.380015 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:09.380035 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.381726 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.381762 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:09.386616 (+ 4854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:09.386639 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.398702 (+ 12063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.398711 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:09.399001 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.399082 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:58:09.399820 (+ 738us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:58:09.399834 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 02:58:09.399927 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 02:58:09.400104 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279}
I20260929 02:58:09.401355 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:09.379059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:54315 (local address 127.8.208.67:34935)
0929 02:58:09.379264 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:09.379272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:09.379293 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:58:09.379363 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:09.379369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:09.379458 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:09.379572 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:09.380240 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.381401 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.388418 (+ 7017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:09.389143 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:09.389324 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:09.399382 (+ 10058us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:09.399560 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:09.399758 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:09.400868 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0929 02:58:09.400963 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260929 02:58:09.402655 11300 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 02:58:11.018991 10765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:11.019194 10765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:11.023546 10765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4: Sending DeleteTablet for 3 replicas of tablet 84ea3b53e33d4939acf7919220c727ce
I20260929 02:58:11.024386 10903 tablet_service.cc:1514] Processing DeleteTablet for tablet 84ea3b53e33d4939acf7919220c727ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:11 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:56856
I20260929 02:58:11.024506 11041 tablet_service.cc:1514] Processing DeleteTablet for tablet 84ea3b53e33d4939acf7919220c727ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:11 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:41074
I20260929 02:58:11.024605 11181 tablet_service.cc:1514] Processing DeleteTablet for tablet 84ea3b53e33d4939acf7919220c727ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:11 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:35306
I20260929 02:58:11.037233 11360 tablet_replica.cc:330] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b: stopping tablet replica
I20260929 02:58:11.037398 11358 tablet_replica.cc:330] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb: stopping tablet replica
I20260929 02:58:11.037621 11359 tablet_replica.cc:330] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a: stopping tablet replica
I20260929 02:58:11.037652 11360 raft_consensus.cc:2229] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:11.037650 11358 raft_consensus.cc:2229] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:11.037973 11358 raft_consensus.cc:2258] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:11.038340 11359 raft_consensus.cc:2229] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:11.038602 11360 raft_consensus.cc:2258] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:11.038655 11359 raft_consensus.cc:2258] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:11.039047 11358 ts_tablet_manager.cc:1905] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:11.039639 11360 ts_tablet_manager.cc:1905] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:11.039831 11359 ts_tablet_manager.cc:1905] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:11.046622 11359 ts_tablet_manager.cc:1918] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:58:11.046718 11360 ts_tablet_manager.cc:1918] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:58:11.046788 11359 log.cc:1196] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/84ea3b53e33d4939acf7919220c727ce
I20260929 02:58:11.046855 11360 log.cc:1196] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/84ea3b53e33d4939acf7919220c727ce
I20260929 02:58:11.047415 11360 ts_tablet_manager.cc:1939] T 84ea3b53e33d4939acf7919220c727ce P 802a4d73449747d69f6c3cacbf95886b: Deleting consensus metadata
I20260929 02:58:11.047432 11359 ts_tablet_manager.cc:1939] T 84ea3b53e33d4939acf7919220c727ce P 93235c192b3d43f0923782e8288ce61a: Deleting consensus metadata
I20260929 02:58:11.047569 11358 ts_tablet_manager.cc:1918] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:58:11.047720 11358 log.cc:1196] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/84ea3b53e33d4939acf7919220c727ce
I20260929 02:58:11.048161 11358 ts_tablet_manager.cc:1939] T 84ea3b53e33d4939acf7919220c727ce P 35dd9a6f086842cbb07f628dc0b94feb: Deleting consensus metadata
I20260929 02:58:11.048697 10753 catalog_manager.cc:4841] TS 802a4d73449747d69f6c3cacbf95886b (127.8.208.67:34935): tablet 84ea3b53e33d4939acf7919220c727ce (table test-table [id=aeffc93e2e8c4a56af84dd56e3cd8da0]) successfully deleted
I20260929 02:58:11.048693 10750 catalog_manager.cc:4841] TS 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093): tablet 84ea3b53e33d4939acf7919220c727ce (table test-table [id=aeffc93e2e8c4a56af84dd56e3cd8da0]) successfully deleted
I20260929 02:58:11.049249 10752 catalog_manager.cc:4841] TS 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035): tablet 84ea3b53e33d4939acf7919220c727ce (table test-table [id=aeffc93e2e8c4a56af84dd56e3cd8da0]) successfully deleted
Sep 29 02:58:11 dist-test-slave-m1rv krb5kdc[10721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650691, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:58:11 dist-test-slave-m1rv krb5kdc[10721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650691, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:11.087098 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:11.073516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60082 (local address 127.8.208.126:33131)
0929 02:58:11.074081 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:11.074086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:11.074109 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:58:11.074266 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:11.074269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:11.074332 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:11.074434 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:11.075610 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:11.076337 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:11.077593 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:11.077983 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:11.078079 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:11.081739 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:11.081743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:11.081747 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:11.081775 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:11.084468 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:11.085115 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:11.085122 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:11.085125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:11.085322 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:11.085712 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:11.085717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:11.085721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:11.085986 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:11.086229 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:11.086511 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 02:58:11.086725 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":327,"threads_started":1}
I20260929 02:58:11.090698 10765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:58:11.091317 10765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:11.097355 10903 tablet_service.cc:1467] Processing CreateTablet for tablet 79a0c461d623429bbac2873f74dee6bb (DEFAULT_TABLE table=test-table [id=1175f95a2ec1494bba632fafad9416cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:11.097781 10903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79a0c461d623429bbac2873f74dee6bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:11.097822 11181 tablet_service.cc:1467] Processing CreateTablet for tablet 79a0c461d623429bbac2873f74dee6bb (DEFAULT_TABLE table=test-table [id=1175f95a2ec1494bba632fafad9416cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:11.098258 11181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79a0c461d623429bbac2873f74dee6bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:11.098914 11041 tablet_service.cc:1467] Processing CreateTablet for tablet 79a0c461d623429bbac2873f74dee6bb (DEFAULT_TABLE table=test-table [id=1175f95a2ec1494bba632fafad9416cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:11.099325 11041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79a0c461d623429bbac2873f74dee6bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:11.104524 11370 tablet_bootstrap.cc:492] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb: Bootstrap starting.
I20260929 02:58:11.105803 11371 tablet_bootstrap.cc:492] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a: Bootstrap starting.
I20260929 02:58:11.106109 11369 tablet_bootstrap.cc:492] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b: Bootstrap starting.
I20260929 02:58:11.107159 11370 tablet_bootstrap.cc:654] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:11.107445 11371 tablet_bootstrap.cc:654] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:11.107713 11369 tablet_bootstrap.cc:654] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:11.109145 11370 tablet_bootstrap.cc:492] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb: No bootstrap required, opened a new log
I20260929 02:58:11.109346 11370 ts_tablet_manager.cc:1397] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 02:58:11.109704 11369 tablet_bootstrap.cc:492] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b: No bootstrap required, opened a new log
I20260929 02:58:11.109843 11369 ts_tablet_manager.cc:1397] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 02:58:11.110065 11370 raft_consensus.cc:348] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.110251 11370 raft_consensus.cc:374] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:11.110356 11370 raft_consensus.cc:729] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dd9a6f086842cbb07f628dc0b94feb, State: Initialized, Role: FOLLOWER
I20260929 02:58:11.110538 11369 raft_consensus.cc:348] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.110605 11370 consensus_queue.cc:260] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.110754 11369 raft_consensus.cc:374] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:11.110842 11369 raft_consensus.cc:729] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802a4d73449747d69f6c3cacbf95886b, State: Initialized, Role: FOLLOWER
I20260929 02:58:11.111089 11369 consensus_queue.cc:260] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.111696 11370 ts_tablet_manager.cc:1428] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:58:11.112753 11371 tablet_bootstrap.cc:492] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a: No bootstrap required, opened a new log
I20260929 02:58:11.112900 11371 ts_tablet_manager.cc:1397] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260929 02:58:11.112934 11369 ts_tablet_manager.cc:1428] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:11.113723 11371 raft_consensus.cc:348] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.113962 11371 raft_consensus.cc:374] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:11.114058 11371 raft_consensus.cc:729] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93235c192b3d43f0923782e8288ce61a, State: Initialized, Role: FOLLOWER
I20260929 02:58:11.114282 11371 consensus_queue.cc:260] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.114845 11371 ts_tablet_manager.cc:1428] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260929 02:58:11.228271 11345 raft_consensus.cc:482] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:11.228528 11345 raft_consensus.cc:504] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.229254 11345 leader_election.cc:288] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035), 802a4d73449747d69f6c3cacbf95886b (127.8.208.67:34935)
I20260929 02:58:11.229728 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79a0c461d623429bbac2873f74dee6bb" candidate_uuid: "93235c192b3d43f0923782e8288ce61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dd9a6f086842cbb07f628dc0b94feb" is_pre_election: true
I20260929 02:58:11.229806 11205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79a0c461d623429bbac2873f74dee6bb" candidate_uuid: "93235c192b3d43f0923782e8288ce61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "802a4d73449747d69f6c3cacbf95886b" is_pre_election: true
I20260929 02:58:11.229966 10923 raft_consensus.cc:2454] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93235c192b3d43f0923782e8288ce61a in term 0.
I20260929 02:58:11.230078 11205 raft_consensus.cc:2454] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93235c192b3d43f0923782e8288ce61a in term 0.
I20260929 02:58:11.230525 10995 leader_election.cc:302] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dd9a6f086842cbb07f628dc0b94feb, 93235c192b3d43f0923782e8288ce61a; no voters:
I20260929 02:58:11.230821 11345 raft_consensus.cc:2781] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:11.230943 11345 raft_consensus.cc:482] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:11.231033 11345 raft_consensus.cc:3037] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:11.233033 11345 raft_consensus.cc:504] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.233542 11345 leader_election.cc:288] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [CANDIDATE]: Term 1 election: Requested vote from peers 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035), 802a4d73449747d69f6c3cacbf95886b (127.8.208.67:34935)
I20260929 02:58:11.234068 11205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79a0c461d623429bbac2873f74dee6bb" candidate_uuid: "93235c192b3d43f0923782e8288ce61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "802a4d73449747d69f6c3cacbf95886b"
I20260929 02:58:11.234023 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79a0c461d623429bbac2873f74dee6bb" candidate_uuid: "93235c192b3d43f0923782e8288ce61a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dd9a6f086842cbb07f628dc0b94feb"
I20260929 02:58:11.234263 11205 raft_consensus.cc:3037] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:11.234279 10923 raft_consensus.cc:3037] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:11.237272 11205 raft_consensus.cc:2454] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93235c192b3d43f0923782e8288ce61a in term 1.
I20260929 02:58:11.237272 10923 raft_consensus.cc:2454] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93235c192b3d43f0923782e8288ce61a in term 1.
I20260929 02:58:11.237681 10996 leader_election.cc:302] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 802a4d73449747d69f6c3cacbf95886b, 93235c192b3d43f0923782e8288ce61a; no voters:
I20260929 02:58:11.237974 11345 raft_consensus.cc:2781] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:11.238117 11345 raft_consensus.cc:686] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 1 LEADER]: Becoming Leader. State: Replica: 93235c192b3d43f0923782e8288ce61a, State: Running, Role: LEADER
I20260929 02:58:11.238400 11345 consensus_queue.cc:237] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.241055 10765 catalog_manager.cc:5495] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a reported cstate change: term changed from 0 to 1, leader changed from <none> to 93235c192b3d43f0923782e8288ce61a (127.8.208.66). New cstate: current_term: 1 leader_uuid: "93235c192b3d43f0923782e8288ce61a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } health_report { overall_health: UNKNOWN } } }
I20260929 02:58:11.267380 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:11.262825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41088 (local address 127.8.208.66:40093)
0929 02:58:11.263297 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:11.263301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:11.263320 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:58:11.263385 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:11.263388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:11.263442 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:11.263519 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:11.263815 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:11.264563 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:11.265882 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:11.266282 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:11.266381 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:11.266754 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:11.266851 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:11.266916 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:11.267035 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 02:58:11.267090 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":265,"threads_started":1}
I20260929 02:58:11.280635 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:11.276031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:54437 (local address 127.8.208.66:40093)
0929 02:58:11.276263 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:11.276274 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:11.276631 (+ 357us) server_negotiation.cc:407] Connection header received
0929 02:58:11.276762 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:11.276802 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:11.276867 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:11.276949 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:11.277363 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:11.278051 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:11.279301 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:11.279704 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:11.279799 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:11.280124 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:11.280206 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:11.280269 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:11.280386 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 02:58:11.280438 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:58:11.280768 11381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:11.275936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:40093 (local address 127.8.208.66:54437)
0929 02:58:11.276539 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:11.276554 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:11.276660 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:11.276969 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:11.276972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:11.276980 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:58:11.277229 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:11.277237 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:11.278158 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:11.278161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:11.279183 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:11.279194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:11.279853 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:11.279857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:11.279979 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:11.280011 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:58:11.280282 (+ 271us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:58:11.280290 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:58:11.280345 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 02:58:11.280449 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":1,"thread_start_us":365,"threads_started":1}
I20260929 02:58:11.288121 11205 raft_consensus.cc:1261] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Refusing update from remote peer 93235c192b3d43f0923782e8288ce61a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:11.288141 10923 raft_consensus.cc:1261] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Refusing update from remote peer 93235c192b3d43f0923782e8288ce61a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:11.288585 11340 consensus_queue.cc:1035] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [LEADER]: Connected to new peer: Peer: permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:11.288909 11337 consensus_queue.cc:1035] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:11.334319 10765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:11.334578 10765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:11.337496 10765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4: Sending DeleteTablet for 3 replicas of tablet 79a0c461d623429bbac2873f74dee6bb
I20260929 02:58:11.338308 10903 tablet_service.cc:1514] Processing DeleteTablet for tablet 79a0c461d623429bbac2873f74dee6bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:11 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:56856
I20260929 02:58:11.338392 11041 tablet_service.cc:1514] Processing DeleteTablet for tablet 79a0c461d623429bbac2873f74dee6bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:11 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:41074
I20260929 02:58:11.338536 11181 tablet_service.cc:1514] Processing DeleteTablet for tablet 79a0c461d623429bbac2873f74dee6bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:11 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:35306
I20260929 02:58:11.338615 11358 tablet_replica.cc:330] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb: stopping tablet replica
I20260929 02:58:11.338804 11359 tablet_replica.cc:330] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a: stopping tablet replica
I20260929 02:58:11.338810 11358 raft_consensus.cc:2229] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:11.339001 11358 raft_consensus.cc:2258] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:11.339020 11359 raft_consensus.cc:2229] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:11.339540 11359 raft_consensus.cc:2258] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:11.339845 11360 tablet_replica.cc:330] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b: stopping tablet replica
I20260929 02:58:11.340050 11360 raft_consensus.cc:2229] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:11.340088 11358 ts_tablet_manager.cc:1905] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:11.340238 11360 raft_consensus.cc:2258] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:11.341240 11359 ts_tablet_manager.cc:1905] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:11.341240 11360 ts_tablet_manager.cc:1905] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:11.346549 10765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 02:58:11.346915 11358 ts_tablet_manager.cc:1918] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:11.347080 11358 log.cc:1196] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/79a0c461d623429bbac2873f74dee6bb
W20260929 02:58:11.347254 10765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:11.347500 11358 ts_tablet_manager.cc:1939] T 79a0c461d623429bbac2873f74dee6bb P 35dd9a6f086842cbb07f628dc0b94feb: Deleting consensus metadata
I20260929 02:58:11.348501 10752 catalog_manager.cc:4841] TS 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035): tablet 79a0c461d623429bbac2873f74dee6bb (table test-table [id=1175f95a2ec1494bba632fafad9416cf]) successfully deleted
I20260929 02:58:11.349419 11360 ts_tablet_manager.cc:1918] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:11.349572 11360 log.cc:1196] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/79a0c461d623429bbac2873f74dee6bb
I20260929 02:58:11.349982 11360 ts_tablet_manager.cc:1939] T 79a0c461d623429bbac2873f74dee6bb P 802a4d73449747d69f6c3cacbf95886b: Deleting consensus metadata
I20260929 02:58:11.351078 10753 catalog_manager.cc:4841] TS 802a4d73449747d69f6c3cacbf95886b (127.8.208.67:34935): tablet 79a0c461d623429bbac2873f74dee6bb (table test-table [id=1175f95a2ec1494bba632fafad9416cf]) successfully deleted
I20260929 02:58:11.354202 10903 tablet_service.cc:1467] Processing CreateTablet for tablet db0944eb285d4fc2800e1936222a5499 (DEFAULT_TABLE table=test-table [id=b3db004194134d1f899d1332fd93a57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:11.354264 11359 ts_tablet_manager.cc:1918] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:11.354601 10903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db0944eb285d4fc2800e1936222a5499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:11.354718 11181 tablet_service.cc:1467] Processing CreateTablet for tablet db0944eb285d4fc2800e1936222a5499 (DEFAULT_TABLE table=test-table [id=b3db004194134d1f899d1332fd93a57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:11.355099 11181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db0944eb285d4fc2800e1936222a5499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:11.359365 11369 tablet_bootstrap.cc:492] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b: Bootstrap starting.
I20260929 02:58:11.359982 11041 tablet_service.cc:1467] Processing CreateTablet for tablet db0944eb285d4fc2800e1936222a5499 (DEFAULT_TABLE table=test-table [id=b3db004194134d1f899d1332fd93a57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:11.360404 11041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db0944eb285d4fc2800e1936222a5499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:11.360611 11370 tablet_bootstrap.cc:492] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb: Bootstrap starting.
I20260929 02:58:11.360948 11369 tablet_bootstrap.cc:654] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:11.362164 11370 tablet_bootstrap.cc:654] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:11.367733 11359 log.cc:1196] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/79a0c461d623429bbac2873f74dee6bb
I20260929 02:58:11.368294 11359 ts_tablet_manager.cc:1939] T 79a0c461d623429bbac2873f74dee6bb P 93235c192b3d43f0923782e8288ce61a: Deleting consensus metadata
I20260929 02:58:11.369462 11371 tablet_bootstrap.cc:492] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a: Bootstrap starting.
I20260929 02:58:11.371459 11371 tablet_bootstrap.cc:654] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:11.372468 11370 tablet_bootstrap.cc:492] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb: No bootstrap required, opened a new log
I20260929 02:58:11.372630 11370 ts_tablet_manager.cc:1397] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260929 02:58:11.373378 11370 raft_consensus.cc:348] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.373629 11370 raft_consensus.cc:374] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:11.373718 11370 raft_consensus.cc:729] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dd9a6f086842cbb07f628dc0b94feb, State: Initialized, Role: FOLLOWER
I20260929 02:58:11.373988 11370 consensus_queue.cc:260] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.374687 11369 tablet_bootstrap.cc:492] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b: No bootstrap required, opened a new log
I20260929 02:58:11.375242 11370 ts_tablet_manager.cc:1428] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:11.375435 11369 ts_tablet_manager.cc:1397] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.000s
I20260929 02:58:11.376519 11369 raft_consensus.cc:348] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.376778 11369 raft_consensus.cc:374] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:11.377286 11369 raft_consensus.cc:729] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802a4d73449747d69f6c3cacbf95886b, State: Initialized, Role: FOLLOWER
I20260929 02:58:11.377534 11369 consensus_queue.cc:260] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.378232 11369 ts_tablet_manager.cc:1428] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:11.378948 10750 catalog_manager.cc:4841] TS 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093): tablet 79a0c461d623429bbac2873f74dee6bb (table test-table [id=1175f95a2ec1494bba632fafad9416cf]) successfully deleted
I20260929 02:58:11.379778 11335 raft_consensus.cc:482] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:11.379982 11335 raft_consensus.cc:504] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.380889 11335 leader_election.cc:288] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035), 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093)
I20260929 02:58:11.381246 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db0944eb285d4fc2800e1936222a5499" candidate_uuid: "802a4d73449747d69f6c3cacbf95886b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dd9a6f086842cbb07f628dc0b94feb" is_pre_election: true
I20260929 02:58:11.381491 10922 raft_consensus.cc:2454] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 802a4d73449747d69f6c3cacbf95886b in term 0.
I20260929 02:58:11.381887 11134 leader_election.cc:302] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dd9a6f086842cbb07f628dc0b94feb, 802a4d73449747d69f6c3cacbf95886b; no voters:
I20260929 02:58:11.381824 11063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db0944eb285d4fc2800e1936222a5499" candidate_uuid: "802a4d73449747d69f6c3cacbf95886b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93235c192b3d43f0923782e8288ce61a" is_pre_election: true
I20260929 02:58:11.382148 11335 raft_consensus.cc:2781] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:11.382299 11335 raft_consensus.cc:482] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:11.382418 11335 raft_consensus.cc:3037] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 0 FOLLOWER]: Advancing to term 1
W20260929 02:58:11.382654 11132 leader_election.cc:341] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093): Illegal state: must be running to vote when last-logged opid is not known
I20260929 02:58:11.384708 11335 raft_consensus.cc:504] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.385226 11335 leader_election.cc:288] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [CANDIDATE]: Term 1 election: Requested vote from peers 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035), 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093)
I20260929 02:58:11.385746 11063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db0944eb285d4fc2800e1936222a5499" candidate_uuid: "802a4d73449747d69f6c3cacbf95886b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93235c192b3d43f0923782e8288ce61a"
I20260929 02:58:11.385794 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db0944eb285d4fc2800e1936222a5499" candidate_uuid: "802a4d73449747d69f6c3cacbf95886b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dd9a6f086842cbb07f628dc0b94feb"
I20260929 02:58:11.386108 10922 raft_consensus.cc:3037] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb [term 0 FOLLOWER]: Advancing to term 1
W20260929 02:58:11.386389 11132 leader_election.cc:341] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093): Illegal state: must be running to vote when last-logged opid is not known
I20260929 02:58:11.388774 11371 tablet_bootstrap.cc:492] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a: No bootstrap required, opened a new log
I20260929 02:58:11.388960 11371 ts_tablet_manager.cc:1397] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.008s
I20260929 02:58:11.389200 10922 raft_consensus.cc:2454] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 802a4d73449747d69f6c3cacbf95886b in term 1.
I20260929 02:58:11.389636 11134 leader_election.cc:302] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 35dd9a6f086842cbb07f628dc0b94feb, 802a4d73449747d69f6c3cacbf95886b; no voters: 93235c192b3d43f0923782e8288ce61a
I20260929 02:58:11.389945 11335 raft_consensus.cc:2781] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:11.389889 11371 raft_consensus.cc:348] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.390080 11335 raft_consensus.cc:686] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 1 LEADER]: Becoming Leader. State: Replica: 802a4d73449747d69f6c3cacbf95886b, State: Running, Role: LEADER
I20260929 02:58:11.390159 11371 raft_consensus.cc:374] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:11.390285 11371 raft_consensus.cc:729] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93235c192b3d43f0923782e8288ce61a, State: Initialized, Role: FOLLOWER
I20260929 02:58:11.390340 11335 consensus_queue.cc:237] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.390544 11371 consensus_queue.cc:260] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } }
I20260929 02:58:11.392057 11371 ts_tablet_manager.cc:1428] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260929 02:58:11.394245 10765 catalog_manager.cc:5495] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b reported cstate change: term changed from 0 to 1, leader changed from <none> to 802a4d73449747d69f6c3cacbf95886b (127.8.208.67). New cstate: current_term: 1 leader_uuid: "802a4d73449747d69f6c3cacbf95886b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "802a4d73449747d69f6c3cacbf95886b" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34935 } health_report { overall_health: HEALTHY } } }
I20260929 02:58:11.426849 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:11.417195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35330 (local address 127.8.208.67:34935)
0929 02:58:11.421629 (+ 4434us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:11.421633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:11.421653 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:58:11.421709 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:11.421712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:11.421772 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:11.421868 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:11.422363 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:11.423122 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:11.424994 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:11.425462 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:11.425557 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:11.426095 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:11.426214 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:11.426283 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:11.426432 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 02:58:11.426491 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4322,"thread_start_us":4200,"threads_started":1}
I20260929 02:58:11.435577 10922 raft_consensus.cc:1261] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Refusing update from remote peer 802a4d73449747d69f6c3cacbf95886b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:11.435556 11063 raft_consensus.cc:3037] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:11.436901 11350 consensus_queue.cc:1035] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dd9a6f086842cbb07f628dc0b94feb" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:11.439198 11063 raft_consensus.cc:1261] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Refusing update from remote peer 802a4d73449747d69f6c3cacbf95886b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:11.439795 11350 consensus_queue.cc:1035] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [LEADER]: Connected to new peer: Peer: permanent_uuid: "93235c192b3d43f0923782e8288ce61a" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 02:58:11.460038 11384 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 02:58:13.030560 10765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:13.030886 10765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:13.035434 10765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc8c58f7bb864ca89f807896daef44e4: Sending DeleteTablet for 3 replicas of tablet db0944eb285d4fc2800e1936222a5499
I20260929 02:58:13.036221 11041 tablet_service.cc:1514] Processing DeleteTablet for tablet db0944eb285d4fc2800e1936222a5499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:13 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:41074
I20260929 02:58:13.036233 10903 tablet_service.cc:1514] Processing DeleteTablet for tablet db0944eb285d4fc2800e1936222a5499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:13 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:56856
I20260929 02:58:13.036456 11181 tablet_service.cc:1514] Processing DeleteTablet for tablet db0944eb285d4fc2800e1936222a5499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:13 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:35306
I20260929 02:58:13.036890 11416 tablet_replica.cc:330] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb: stopping tablet replica
I20260929 02:58:13.037128 11416 raft_consensus.cc:2229] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:13.037448 11416 raft_consensus.cc:2258] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:13.037569 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 10837
I20260929 02:58:13.038625 11416 ts_tablet_manager.cc:1905] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:13.047770 11416 ts_tablet_manager.cc:1918] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:58:13.047986 11416 log.cc:1196] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/db0944eb285d4fc2800e1936222a5499
I20260929 02:58:13.048576 11416 ts_tablet_manager.cc:1939] T db0944eb285d4fc2800e1936222a5499 P 35dd9a6f086842cbb07f628dc0b94feb: Deleting consensus metadata
I20260929 02:58:13.048990 11417 tablet_replica.cc:330] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b: stopping tablet replica
I20260929 02:58:13.049394 11417 raft_consensus.cc:2229] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:13.049897 11417 raft_consensus.cc:2258] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:13.049965 10752 catalog_manager.cc:4841] TS 35dd9a6f086842cbb07f628dc0b94feb (127.8.208.65:41035): tablet db0944eb285d4fc2800e1936222a5499 (table test-table [id=b3db004194134d1f899d1332fd93a57f]) successfully deleted
I20260929 02:58:13.051192 11417 ts_tablet_manager.cc:1905] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:13.054797 11415 tablet_replica.cc:330] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a: stopping tablet replica
I20260929 02:58:13.054999 11415 raft_consensus.cc:2229] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:13.055233 11415 raft_consensus.cc:2258] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:13.056245 11415 ts_tablet_manager.cc:1905] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:13.058220 11417 ts_tablet_manager.cc:1918] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:58:13.058379 11417 log.cc:1196] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/db0944eb285d4fc2800e1936222a5499
I20260929 02:58:13.058799 11417 ts_tablet_manager.cc:1939] T db0944eb285d4fc2800e1936222a5499 P 802a4d73449747d69f6c3cacbf95886b: Deleting consensus metadata
I20260929 02:58:13.059772 10753 catalog_manager.cc:4841] TS 802a4d73449747d69f6c3cacbf95886b (127.8.208.67:34935): tablet db0944eb285d4fc2800e1936222a5499 (table test-table [id=b3db004194134d1f899d1332fd93a57f]) successfully deleted
I20260929 02:58:13.063118 11415 ts_tablet_manager.cc:1918] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:58:13.063264 11415 log.cc:1196] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/db0944eb285d4fc2800e1936222a5499
I20260929 02:58:13.063668 11415 ts_tablet_manager.cc:1939] T db0944eb285d4fc2800e1936222a5499 P 93235c192b3d43f0923782e8288ce61a: Deleting consensus metadata
I20260929 02:58:13.064550 10750 catalog_manager.cc:4841] TS 93235c192b3d43f0923782e8288ce61a (127.8.208.66:40093): tablet db0944eb285d4fc2800e1936222a5499 (table test-table [id=b3db004194134d1f899d1332fd93a57f]) successfully deleted
Sep 29 02:58:13 dist-test-slave-m1rv krb5kdc[10721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650691, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:58:13.073982 11419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:13.039956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56882 (local address 127.8.208.65:41035)
0929 02:58:13.053661 (+ 13705us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:13.053683 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:13.054168 (+ 485us) server_negotiation.cc:407] Connection header received
0929 02:58:13.054381 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:13.054388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:13.054556 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:13.054672 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:13.056278 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:13.057459 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:13.059325 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:13.059887 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:13.060007 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:13.068736 (+ 8729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:13.068753 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:13.068770 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:13.068804 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:13.071510 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:13.072088 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:13.072095 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:13.072098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:13.072219 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:13.072546 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:13.072550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:13.072552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:13.072915 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:13.073120 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:13.073421 (+ 301us) server_negotiation.cc:299] Negotiation successful
0929 02:58:13.073582 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13512,"thread_start_us":13418,"threads_started":1}
W20260929 02:58:13.074662 10968 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56882
W20260929 02:58:13.075263 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 10837: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:58:13.075520 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 10837
I20260929 02:58:13.097609 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 10977
Sep 29 02:58:13 dist-test-slave-m1rv krb5kdc[10721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650691, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 02:58:13.111672 11420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:13.098338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41094 (local address 127.8.208.66:40093)
0929 02:58:13.098885 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:13.098889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:13.098923 (+ 34us) server_negotiation.cc:407] Connection header received
0929 02:58:13.098987 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:13.098991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:13.099046 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:13.099132 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:13.100304 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:13.101032 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:13.102242 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:13.102620 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:13.102709 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:13.106681 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:13.106693 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:13.106708 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:13.106742 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:13.109549 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:13.110101 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:13.110108 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:13.110111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:13.110253 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:13.110549 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:13.110552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:13.110554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:13.110854 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:13.111083 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:13.111264 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 02:58:13.111419 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":261,"threads_started":1}
W20260929 02:58:13.112311 11107 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41094
W20260929 02:58:13.112901 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 10977: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:58:13.113035 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 10977
I20260929 02:58:13.137401 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 11116
Sep 29 02:58:13 dist-test-slave-m1rv krb5kdc[10721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650691, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 02:58:13.151338 11421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:13.138097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35342 (local address 127.8.208.67:34935)
0929 02:58:13.138575 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:13.138579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:13.138595 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:58:13.138648 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:13.138652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:13.138713 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:13.138803 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:13.139997 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:13.140737 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:13.142040 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:13.142416 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:13.142508 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:13.146546 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:13.146560 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:13.146577 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:13.146605 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:13.149236 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:13.149765 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:13.149772 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:13.149774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:13.149904 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:13.150202 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:13.150205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:13.150207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:13.150508 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:13.150723 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:13.150914 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 02:58:13.151068 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":301,"threads_started":1}
W20260929 02:58:13.151978 11246 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35342
W20260929 02:58:13.152518 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 11116: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:58:13.152688 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 11116
I20260929 02:58:13.174741 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 10734
I20260929 02:58:13.175899 10806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:13.290874 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 10734
2026-09-29T02:58:13Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8072 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:58:13 dist-test-slave-m1rv krb5kdc[11425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:58:13 dist-test-slave-m1rv krb5kdc[11425](info): set up 2 sockets
Sep 29 02:58:13 dist-test-slave-m1rv krb5kdc[11425](info): commencing operation
krb5kdc: starting...
W20260929 02:58:14.377374 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:58:14 dist-test-slave-m1rv krb5kdc[11425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:58:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:58:14.566205 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:45333
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:45333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:14.878322 11438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:14.878611 11438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:14.878701 11438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:14.885854 11438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:14.885959 11438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:14.886005 11438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:14.886055 11438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:14.886096 11438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:14.895853 11438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:45333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:45333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:14.897101 11438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:14.898373 11438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:14.909653 11446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:14.909741 11443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:14.909742 11444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:14.910667 11438 server_base.cc:1034] running on GCE node
I20260929 02:58:14.911392 11438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:14.912709 11438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:14.913920 11438 hybrid_clock.cc:648] HybridClock initialized: now 1790650694913877 us; error 52 us; skew 500 ppm
I20260929 02:58:14.914255 11438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:14 dist-test-slave-m1rv krb5kdc[11425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650694, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:14.917232 11438 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:58:14.918658 11438 webserver.cc:460] Webserver started at http://127.8.208.126:44117/ using document root <none> and password file <none>
I20260929 02:58:14.919319 11438 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:14.919463 11438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:14.919754 11438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:14.922632 11438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "ff264c7c02714568a05f522d98f97e82"
format_stamp: "Formatted at 2026-09-29 02:58:14 on dist-test-slave-m1rv"
server_key: "cbbeb8173feac3d46e9aec2153b82328"
server_key_iv: "40934e4599bfb63461e45135bf9875d5"
server_key_version: "encryptionkey@0"
I20260929 02:58:14.923415 11438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "ff264c7c02714568a05f522d98f97e82"
format_stamp: "Formatted at 2026-09-29 02:58:14 on dist-test-slave-m1rv"
server_key: "cbbeb8173feac3d46e9aec2153b82328"
server_key_iv: "40934e4599bfb63461e45135bf9875d5"
server_key_version: "encryptionkey@0"
I20260929 02:58:14.928409 11438 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:58:14.931689 11453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:14.932927 11438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:14.933096 11438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "ff264c7c02714568a05f522d98f97e82"
format_stamp: "Formatted at 2026-09-29 02:58:14 on dist-test-slave-m1rv"
server_key: "cbbeb8173feac3d46e9aec2153b82328"
server_key_iv: "40934e4599bfb63461e45135bf9875d5"
server_key_version: "encryptionkey@0"
I20260929 02:58:14.933346 11438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:14.946806 11438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:14.950479 11438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:14.950743 11438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:14.968287 11438 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:45333
I20260929 02:58:14.968317 11504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:45333 every 8 connection(s)
I20260929 02:58:14.969643 11438 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:14.972437 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 11438
I20260929 02:58:14.972697 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:14.973568 9025 external_mini_cluster.cc:1428] Setting key e194923d15c0e9fe44b0c60b79920902
I20260929 02:58:14.974167 11505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:14.989209 11505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82: Bootstrap starting.
Sep 29 02:58:14 dist-test-slave-m1rv krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:14.993330 11505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:14.994755 11505 log.cc:824] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:14.998262 11505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82: No bootstrap required, opened a new log
I20260929 02:58:14.999574 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:14.975713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35126 (local address 127.8.208.126:45333)
0929 02:58:14.976465 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:14.976476 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:14.976514 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:58:14.978871 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:14.978892 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:14.979272 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:14.979587 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:14.981589 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:14.982882 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:14.984756 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:14.985502 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:14.985633 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:14.990752 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:14.990766 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:14.990788 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:14.990849 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:14.994958 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:14.995719 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:14.995725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:14.995735 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:14.995916 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:14.996325 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:14.996330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:14.996333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:14.997090 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:14.997429 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:14.997748 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 02:58:14.998169 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":358,"threads_started":1}
I20260929 02:58:15.007139 11505 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff264c7c02714568a05f522d98f97e82" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45333 } }
I20260929 02:58:15.007563 11505 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:15.007651 11505 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff264c7c02714568a05f522d98f97e82, State: Initialized, Role: FOLLOWER
I20260929 02:58:15.008285 11505 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff264c7c02714568a05f522d98f97e82" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45333 } }
I20260929 02:58:15.008603 11505 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:15.008780 11505 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:15.009001 11505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:15.011327 11505 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff264c7c02714568a05f522d98f97e82" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45333 } }
I20260929 02:58:15.011865 11505 leader_election.cc:302] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff264c7c02714568a05f522d98f97e82; no voters:
I20260929 02:58:15.012490 11505 leader_election.cc:288] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:15.012609 11510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:15.013429 11510 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [term 1 LEADER]: Becoming Leader. State: Replica: ff264c7c02714568a05f522d98f97e82, State: Running, Role: LEADER
I20260929 02:58:15.014032 11510 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff264c7c02714568a05f522d98f97e82" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45333 } }
I20260929 02:58:15.014918 11505 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:15.017310 11512 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff264c7c02714568a05f522d98f97e82. Latest consensus state: current_term: 1 leader_uuid: "ff264c7c02714568a05f522d98f97e82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff264c7c02714568a05f522d98f97e82" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45333 } } }
I20260929 02:58:15.017587 11511 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff264c7c02714568a05f522d98f97e82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff264c7c02714568a05f522d98f97e82" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45333 } } }
I20260929 02:58:15.017836 11512 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:15.017863 11511 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:15.020076 11519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:15.024443 11519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:15.032179 11519 catalog_manager.cc:1347] Generated new cluster ID: 0f6d2a5113104ed082454d7d5ae02f99
I20260929 02:58:15.032294 11519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:15.060618 11519 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:15.062505 11519 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:15.072510 11519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff264c7c02714568a05f522d98f97e82: Generated new TSK 0
I20260929 02:58:15.073555 11519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 02:58:15.118863 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:45333
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:15.437961 11531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:15.438261 11531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:15.438419 11531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:15.445545 11531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:15.445691 11531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:15.445876 11531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:58:15.455600 11531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:45333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:15.456903 11531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:15.458240 11531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:15.471565 11537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:15.471764 11536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:15.471598 11539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:15.472625 11531 server_base.cc:1034] running on GCE node
I20260929 02:58:15.473274 11531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:15.474164 11531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:15.475353 11531 hybrid_clock.cc:648] HybridClock initialized: now 1790650695475313 us; error 32 us; skew 500 ppm
I20260929 02:58:15.475622 11531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:15 dist-test-slave-m1rv krb5kdc[11425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650695, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:15.478770 11531 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 02:58:15.480273 11531 webserver.cc:460] Webserver started at http://127.8.208.65:37427/ using document root <none> and password file <none>
I20260929 02:58:15.480967 11531 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:15.481091 11531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:15.481436 11531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:15.484274 11531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "025adeb227d54b099d49a5e43c52aa30"
format_stamp: "Formatted at 2026-09-29 02:58:15 on dist-test-slave-m1rv"
server_key: "03a5d2d3ebe9844c5bb8ba52abf8cbcc"
server_key_iv: "7b4647558fccc0923d35a0841c09cbb3"
server_key_version: "encryptionkey@0"
I20260929 02:58:15.485064 11531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "025adeb227d54b099d49a5e43c52aa30"
format_stamp: "Formatted at 2026-09-29 02:58:15 on dist-test-slave-m1rv"
server_key: "03a5d2d3ebe9844c5bb8ba52abf8cbcc"
server_key_iv: "7b4647558fccc0923d35a0841c09cbb3"
server_key_version: "encryptionkey@0"
I20260929 02:58:15.490180 11531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 02:58:15.493438 11546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:15.494722 11531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:58:15.494879 11531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "025adeb227d54b099d49a5e43c52aa30"
format_stamp: "Formatted at 2026-09-29 02:58:15 on dist-test-slave-m1rv"
server_key: "03a5d2d3ebe9844c5bb8ba52abf8cbcc"
server_key_iv: "7b4647558fccc0923d35a0841c09cbb3"
server_key_version: "encryptionkey@0"
I20260929 02:58:15.495065 11531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:15.544423 11531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:15.548513 11531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:15.548799 11531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:15.549821 11531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:15.551151 11531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:15.551226 11531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:15.551393 11531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:15.551455 11531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:15.585400 11531 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:41547
I20260929 02:58:15.585436 11658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:41547 every 8 connection(s)
I20260929 02:58:15.586891 11531 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:58:15.587601 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 11531
I20260929 02:58:15.587826 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:58:15.588656 9025 external_mini_cluster.cc:1428] Setting key 298ff8f9c1c3ae667192907881d2e1e6
Sep 29 02:58:15 dist-test-slave-m1rv krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650695, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
I20260929 02:58:15.608685 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:15.589761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:52531 (local address 127.8.208.126:45333)
0929 02:58:15.590306 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:15.590312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:15.591380 (+ 1068us) server_negotiation.cc:407] Connection header received
0929 02:58:15.593264 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:15.593270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:15.593363 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:15.593509 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:15.595172 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:15.595894 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:15.597556 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:15.597965 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:15.598113 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:15.601875 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:15.601882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:15.601886 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:15.601930 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:15.605630 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:15.606465 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:15.606469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:15.606471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:15.606579 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:15.606967 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:15.606970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:15.606972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:15.607328 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:15.607437 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:15.608228 (+ 791us) server_negotiation.cc:299] Negotiation successful
0929 02:58:15.608390 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":269,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
I20260929 02:58:15.609925 11662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:15.590423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:45333 (local address 127.8.208.65:52531)
0929 02:58:15.591230 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:15.591275 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:15.592967 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:15.593764 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:15.593773 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:15.594276 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:15.594989 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:15.595012 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:15.596051 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:15.596057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:15.597405 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:15.597417 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:15.598204 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:15.598212 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:15.598484 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:15.599200 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:15.599229 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:15.601616 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:15.605783 (+ 4167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:15.605790 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:15.605803 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:15.606349 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:15.606682 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:15.606685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:15.606688 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:15.606872 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:15.607447 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:15.607453 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:15.607836 (+ 383us) client_negotiation.cc:769] Sending connection context
0929 02:58:15.608142 (+ 306us) client_negotiation.cc:240] Negotiation successful
0929 02:58:15.608589 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":351,"threads_started":1}
I20260929 02:58:15.611518 11659 heartbeater.cc:347] Connected to a master server at 127.8.208.126:45333
I20260929 02:58:15.611829 11659 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:15.612563 11659 heartbeater.cc:511] Master 127.8.208.126:45333 requested a full tablet report, sending...
I20260929 02:58:15.614913 11470 ts_manager.cc:194] Registered new tserver with Master: 025adeb227d54b099d49a5e43c52aa30 (127.8.208.65:41547)
I20260929 02:58:15.617106 11470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:52531
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 02:58:15.631893 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:45333
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:15.947839 11665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:15.948112 11665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:15.948203 11665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:15.955518 11665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:15.955641 11665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:15.955834 11665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:58:15.965339 11665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:45333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:15.966594 11665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:15.967952 11665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:15.981000 11670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:15.980981 11671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:15.981247 11673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:15.981801 11665 server_base.cc:1034] running on GCE node
I20260929 02:58:15.982481 11665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:15.983522 11665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:15.984746 11665 hybrid_clock.cc:648] HybridClock initialized: now 1790650695984686 us; error 56 us; skew 500 ppm
I20260929 02:58:15.985097 11665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:15 dist-test-slave-m1rv krb5kdc[11425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650695, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:15.988102 11665 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 02:58:15.989612 11665 webserver.cc:460] Webserver started at http://127.8.208.66:32965/ using document root <none> and password file <none>
I20260929 02:58:15.990293 11665 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:15.990430 11665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:15.990727 11665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:15.993690 11665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "976484343a1b4073b2e1ee0ae5fb452e"
format_stamp: "Formatted at 2026-09-29 02:58:15 on dist-test-slave-m1rv"
server_key: "b46d0caabba0d6fc9f5438cb67c14d46"
server_key_iv: "a2b122b3ea7861a83f081b200f7d871c"
server_key_version: "encryptionkey@0"
I20260929 02:58:15.994436 11665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "976484343a1b4073b2e1ee0ae5fb452e"
format_stamp: "Formatted at 2026-09-29 02:58:15 on dist-test-slave-m1rv"
server_key: "b46d0caabba0d6fc9f5438cb67c14d46"
server_key_iv: "a2b122b3ea7861a83f081b200f7d871c"
server_key_version: "encryptionkey@0"
I20260929 02:58:15.999859 11665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:58:16.003225 11680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:16.004429 11665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:58:16.004597 11665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "976484343a1b4073b2e1ee0ae5fb452e"
format_stamp: "Formatted at 2026-09-29 02:58:15 on dist-test-slave-m1rv"
server_key: "b46d0caabba0d6fc9f5438cb67c14d46"
server_key_iv: "a2b122b3ea7861a83f081b200f7d871c"
server_key_version: "encryptionkey@0"
I20260929 02:58:16.004806 11665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:16.061072 11665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:16.065079 11665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:16.065397 11665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:16.066486 11665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:16.067839 11665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:16.067938 11665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:16.068075 11665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:16.068145 11665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:16.100621 11665 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:42089
I20260929 02:58:16.100708 11792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:42089 every 8 connection(s)
I20260929 02:58:16.101995 11665 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:58:16.109443 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 11665
I20260929 02:58:16.109647 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:58:16.110162 9025 external_mini_cluster.cc:1428] Setting key 9e472680918afcd6b57e12e14deb676c
Sep 29 02:58:16 dist-test-slave-m1rv krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650695, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:16.124004 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:16.104476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:36363 (local address 127.8.208.126:45333)
0929 02:58:16.104648 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:16.104652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:16.105798 (+ 1146us) server_negotiation.cc:407] Connection header received
0929 02:58:16.107514 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:16.107519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:16.107585 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:16.107675 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:16.109139 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:16.110286 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:16.112164 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:16.112812 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:16.112991 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:16.117037 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:16.117042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:16.117045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:16.117076 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:16.120799 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:16.121536 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:16.121544 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:16.121551 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:16.121698 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:16.122095 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:16.122102 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:16.122108 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:16.122476 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:16.122632 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:16.123294 (+ 662us) server_negotiation.cc:299] Negotiation successful
0929 02:58:16.123542 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:58:16.124861 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:16.104928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:45333 (local address 127.8.208.66:36363)
0929 02:58:16.105623 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:16.105668 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:16.107273 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:16.107896 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:16.107905 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:16.108412 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:16.108966 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:16.108983 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:16.110459 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:16.110464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:16.112030 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:16.112046 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:16.113024 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:16.113029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:16.113274 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:16.113964 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:16.113985 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:16.116804 (+ 2819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:16.120932 (+ 4128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:16.120938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:16.120953 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:16.121425 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:16.121807 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:16.121809 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:16.121812 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:16.121989 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:16.122601 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:16.122606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:16.122915 (+ 309us) client_negotiation.cc:769] Sending connection context
0929 02:58:16.123136 (+ 221us) client_negotiation.cc:240] Negotiation successful
0929 02:58:16.123524 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":301,"threads_started":1}
I20260929 02:58:16.126389 11793 heartbeater.cc:347] Connected to a master server at 127.8.208.126:45333
I20260929 02:58:16.126745 11793 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:16.127645 11793 heartbeater.cc:511] Master 127.8.208.126:45333 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
I20260929 02:58:16.129529 11470 ts_manager.cc:194] Registered new tserver with Master: 976484343a1b4073b2e1ee0ae5fb452e (127.8.208.66:42089)
I20260929 02:58:16.130298 11470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:36363
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 02:58:16.153597 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:45333
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:16.477581 11798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:16.477931 11798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:16.478106 11798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:16.485260 11798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:16.485435 11798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:16.485631 11798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:58:16.495325 11798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:45333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:16.496706 11798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:16.498263 11798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:16.511621 11803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:16.511703 11804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:16.511654 11806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:16.514608 11798 server_base.cc:1034] running on GCE node
I20260929 02:58:16.515311 11798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:16.516352 11798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:16.517581 11798 hybrid_clock.cc:648] HybridClock initialized: now 1790650696517531 us; error 46 us; skew 500 ppm
I20260929 02:58:16.517897 11798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:16 dist-test-slave-m1rv krb5kdc[11425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650696, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:16.521370 11798 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 02:58:16.522744 11798 webserver.cc:460] Webserver started at http://127.8.208.67:37011/ using document root <none> and password file <none>
I20260929 02:58:16.523447 11798 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:16.523545 11798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:16.523787 11798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:16.527968 11798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "7ec2d7cc74bd4cc49ab28e9ef48864b7"
format_stamp: "Formatted at 2026-09-29 02:58:16 on dist-test-slave-m1rv"
server_key: "feb5c8d736a906d5edbc04bcfe966d70"
server_key_iv: "0f94cdaf3b4f6d864da234bf577f7903"
server_key_version: "encryptionkey@0"
I20260929 02:58:16.529142 11798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "7ec2d7cc74bd4cc49ab28e9ef48864b7"
format_stamp: "Formatted at 2026-09-29 02:58:16 on dist-test-slave-m1rv"
server_key: "feb5c8d736a906d5edbc04bcfe966d70"
server_key_iv: "0f94cdaf3b4f6d864da234bf577f7903"
server_key_version: "encryptionkey@0"
I20260929 02:58:16.535807 11798 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 02:58:16.539474 11813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:16.540863 11798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:16.541002 11798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "7ec2d7cc74bd4cc49ab28e9ef48864b7"
format_stamp: "Formatted at 2026-09-29 02:58:16 on dist-test-slave-m1rv"
server_key: "feb5c8d736a906d5edbc04bcfe966d70"
server_key_iv: "0f94cdaf3b4f6d864da234bf577f7903"
server_key_version: "encryptionkey@0"
I20260929 02:58:16.541231 11798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:16.562970 11798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:16.566897 11798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:16.567139 11798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:16.568048 11798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:16.569377 11798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:16.569469 11798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:16.569553 11798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:16.569597 11798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:16.602754 11798 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:35597
I20260929 02:58:16.602828 11925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:35597 every 8 connection(s)
I20260929 02:58:16.604009 11798 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:58:16.610087 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 11798
I20260929 02:58:16.610287 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:58:16.610705 9025 external_mini_cluster.cc:1428] Setting key d49fe2fd1c832cffc7962e96d4bc475a
Sep 29 02:58:16 dist-test-slave-m1rv krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650696, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:16.620023 11659 heartbeater.cc:503] Master 127.8.208.126:45333 was elected leader, sending a full tablet report...
I20260929 02:58:16.623642 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:16.606412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:36521 (local address 127.8.208.126:45333)
0929 02:58:16.606592 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:16.606597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:16.607588 (+ 991us) server_negotiation.cc:407] Connection header received
0929 02:58:16.609555 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:16.609562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:16.609699 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:16.609815 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:16.611449 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:16.612583 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:16.614488 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:16.614873 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:16.615023 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:16.618249 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:16.618253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:16.618257 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:16.618287 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:16.621066 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:16.621820 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:16.621824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:16.621826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:16.621950 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:16.622313 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:16.622315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:16.622317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:16.622553 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:16.622656 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:16.623239 (+ 583us) server_negotiation.cc:299] Negotiation successful
0929 02:58:16.623384 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:58:16.624724 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:16.606766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:45333 (local address 127.8.208.67:36521)
0929 02:58:16.607422 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:16.607459 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:16.609283 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:16.610015 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:16.610024 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:16.610618 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:16.611265 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:16.611291 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:16.612763 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:16.612770 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:16.614365 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:16.614378 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:16.615128 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:16.615133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:16.615316 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:16.615914 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:16.615935 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:16.618065 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:16.621214 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:16.621223 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:16.621237 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:16.621710 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:16.622056 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:16.622059 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:16.622062 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:16.622220 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:16.622650 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:16.622655 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:16.622949 (+ 294us) client_negotiation.cc:769] Sending connection context
0929 02:58:16.623149 (+ 200us) client_negotiation.cc:240] Negotiation successful
0929 02:58:16.623481 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":252,"threads_started":1}
I20260929 02:58:16.626205 11926 heartbeater.cc:347] Connected to a master server at 127.8.208.126:45333
I20260929 02:58:16.626490 11926 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:16.627143 11926 heartbeater.cc:511] Master 127.8.208.126:45333 requested a full tablet report, sending...
I20260929 02:58:16.628398 11469 ts_manager.cc:194] Registered new tserver with Master: 7ec2d7cc74bd4cc49ab28e9ef48864b7 (127.8.208.67:35597)
I20260929 02:58:16.629302 11469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:36521
I20260929 02:58:16.637814 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 02:58:16.666085 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:58:16.663785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.8.208.126:45333)
0929 02:58:16.663959 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:16.663964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:16.664361 (+ 397us) server_negotiation.cc:407] Connection header received
0929 02:58:16.664520 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:16.664524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:16.664592 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:16.664686 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:16.665806 (+ 1120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35138: BlockingRecv error: recv got EOF from 127.0.0.1:35138 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:58:16.669833 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 11531
W20260929 02:58:16.672565 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 11531: Not authorized: Client connection negotiation failed: client connection to 127.8.208.65:41547: server requires authentication, but client does not have Kerberos credentials available
I20260929 02:58:16.672739 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 11531
I20260929 02:58:16.688822 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 11665
W20260929 02:58:16.691584 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 11665: Not authorized: Client connection negotiation failed: client connection to 127.8.208.66:42089: server requires authentication, but client does not have Kerberos credentials available
I20260929 02:58:16.691789 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 11665
I20260929 02:58:16.707255 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 11798
W20260929 02:58:16.710143 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 11798: Not authorized: Client connection negotiation failed: client connection to 127.8.208.67:35597: server requires authentication, but client does not have Kerberos credentials available
I20260929 02:58:16.710291 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 11798
I20260929 02:58:16.725492 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 11438
I20260929 02:58:16.726614 11500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:16.827569 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 11438
2026-09-29T02:58:16Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3534 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:58:16 dist-test-slave-m1rv krb5kdc[11943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:58:16 dist-test-slave-m1rv krb5kdc[11943](info): set up 2 sockets
Sep 29 02:58:16 dist-test-slave-m1rv krb5kdc[11943](info): commencing operation
krb5kdc: starting...
W20260929 02:58:17.909448 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.008s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:58:18 dist-test-slave-m1rv krb5kdc[11943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:58:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:58:18.097281 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:44559
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:45553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:44559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:18.408453 11956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:18.408810 11956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:18.409001 11956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:18.416190 11956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:18.416298 11956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:18.416424 11956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:18.416522 11956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:18.416605 11956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:18.426419 11956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:45553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:44559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:44559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:18.427687 11956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:18.428985 11956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:18.440943 11964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:18.441023 11961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:18.441052 11962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:18.441877 11956 server_base.cc:1034] running on GCE node
I20260929 02:58:18.442657 11956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:18.444132 11956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:18.445341 11956 hybrid_clock.cc:648] HybridClock initialized: now 1790650698445297 us; error 60 us; skew 500 ppm
I20260929 02:58:18.445693 11956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:18 dist-test-slave-m1rv krb5kdc[11943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650698, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:18.459738 11956 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:58:18.461362 11956 webserver.cc:460] Webserver started at http://127.8.208.126:41865/ using document root <none> and password file <none>
I20260929 02:58:18.462081 11956 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:18.462230 11956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:18.462544 11956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:18.465663 11956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "128332fac4d94fab8397041394a4487e"
format_stamp: "Formatted at 2026-09-29 02:58:18 on dist-test-slave-m1rv"
server_key: "3167a3400d2716da3b1eb9d3a6272c87"
server_key_iv: "de3a5951111add0ffa92841c23c02b88"
server_key_version: "encryptionkey@0"
I20260929 02:58:18.466456 11956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "128332fac4d94fab8397041394a4487e"
format_stamp: "Formatted at 2026-09-29 02:58:18 on dist-test-slave-m1rv"
server_key: "3167a3400d2716da3b1eb9d3a6272c87"
server_key_iv: "de3a5951111add0ffa92841c23c02b88"
server_key_version: "encryptionkey@0"
I20260929 02:58:18.471443 11956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:58:18.474733 11971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:18.476037 11956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:58:18.476217 11956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "128332fac4d94fab8397041394a4487e"
format_stamp: "Formatted at 2026-09-29 02:58:18 on dist-test-slave-m1rv"
server_key: "3167a3400d2716da3b1eb9d3a6272c87"
server_key_iv: "de3a5951111add0ffa92841c23c02b88"
server_key_version: "encryptionkey@0"
I20260929 02:58:18.476405 11956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:18.498791 11956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:18.502975 11956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:18.503266 11956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:18.521337 11956 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:44559
I20260929 02:58:18.521356 12022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:44559 every 8 connection(s)
I20260929 02:58:18.522712 11956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:18.525425 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 11956
I20260929 02:58:18.525635 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:18.526268 9025 external_mini_cluster.cc:1428] Setting key 1b4d896a270d3cf0113493f98c0d06ad
I20260929 02:58:18.526312 12023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:18.540993 12023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e: Bootstrap starting.
Sep 29 02:58:18 dist-test-slave-m1rv krb5kdc[11943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:18.544726 12023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:18.546082 12023 log.cc:824] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:18.549279 12023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e: No bootstrap required, opened a new log
I20260929 02:58:18.551270 12026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:18.528067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50292 (local address 127.8.208.126:44559)
0929 02:58:18.528863 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:18.528878 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:18.528954 (+ 76us) server_negotiation.cc:407] Connection header received
0929 02:58:18.531034 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:18.531058 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:18.531554 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:18.531968 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:18.533264 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:18.535200 (+ 1936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:18.536450 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:18.537578 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:18.537759 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:18.542960 (+ 5201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:18.542970 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:18.542995 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:18.543052 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:18.546935 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:18.547686 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:18.547694 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:18.547702 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:18.547891 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:18.548229 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:18.548234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:18.548237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:18.548907 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:18.549155 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:18.549579 (+ 424us) server_negotiation.cc:299] Negotiation successful
0929 02:58:18.550012 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":327,"threads_started":1}
I20260929 02:58:18.555162 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:58:18.557971 12023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128332fac4d94fab8397041394a4487e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 44559 } }
I20260929 02:58:18.558295 12023 raft_consensus.cc:374] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:18.558388 12023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128332fac4d94fab8397041394a4487e, State: Initialized, Role: FOLLOWER
I20260929 02:58:18.558912 12023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128332fac4d94fab8397041394a4487e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 44559 } }
I20260929 02:58:18.559127 12023 raft_consensus.cc:388] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:18.559249 12023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:18.559418 12023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:18.561549 12023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128332fac4d94fab8397041394a4487e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 44559 } }
I20260929 02:58:18.562026 12023 leader_election.cc:302] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 128332fac4d94fab8397041394a4487e; no voters:
I20260929 02:58:18.562666 12023 leader_election.cc:288] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:18.562763 12029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:18.564206 12029 raft_consensus.cc:686] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [term 1 LEADER]: Becoming Leader. State: Replica: 128332fac4d94fab8397041394a4487e, State: Running, Role: LEADER
I20260929 02:58:18.564831 12029 consensus_queue.cc:237] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128332fac4d94fab8397041394a4487e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 44559 } }
I20260929 02:58:18.565019 12023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:18.570940 12030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "128332fac4d94fab8397041394a4487e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128332fac4d94fab8397041394a4487e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 44559 } } }
I20260929 02:58:18.571344 12030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:18.572049 12031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 128332fac4d94fab8397041394a4487e. Latest consensus state: current_term: 1 leader_uuid: "128332fac4d94fab8397041394a4487e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "128332fac4d94fab8397041394a4487e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 44559 } } }
I20260929 02:58:18.572404 12031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:18.577108 12042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:18.583647 12042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:18.594035 12042 catalog_manager.cc:1347] Generated new cluster ID: 691266c5f9254c4fb2cac7367304c5f0
I20260929 02:58:18.594151 12042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:18.599192 12026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:18.591422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50302 (local address 127.8.208.126:44559)
0929 02:58:18.591645 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:18.591654 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:18.592452 (+ 798us) server_negotiation.cc:407] Connection header received
0929 02:58:18.592634 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:18.592641 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:18.592739 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:18.592853 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:18.594106 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:18.594911 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:18.596172 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:18.596595 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:18.596723 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:18.597834 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:18.597839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:18.597847 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:18.598429 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:18.598470 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:18.598497 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:18.598610 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:18.598755 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 02:58:18.598850 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:58:18.613929 12042 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:18.615182 12042 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:18.624223 12042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 128332fac4d94fab8397041394a4487e: Generated new TSK 0
I20260929 02:58:18.625253 12042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:58:18.638074 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 11956
I20260929 02:58:18.639360 12018 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:18.730659 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 11956
2026-09-29T02:58:18Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1897 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:58:18 dist-test-slave-m1rv krb5kdc[12058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:58:18 dist-test-slave-m1rv krb5kdc[12058](info): set up 2 sockets
Sep 29 02:58:18 dist-test-slave-m1rv krb5kdc[12058](info): commencing operation
krb5kdc: starting...
W20260929 02:58:19.809442 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:58:19 dist-test-slave-m1rv krb5kdc[12058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:58:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:58:19.996387 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:43265
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:43265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:20.309921 12071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:20.310264 12071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:20.310410 12071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:20.317559 12071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:20.317690 12071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:20.317799 12071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:20.317901 12071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:20.317975 12071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:20.327582 12071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:43265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:43265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:20.328902 12071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:20.330194 12071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:20.341622 12076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:20.341710 12079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:20.341876 12077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:20.342406 12071 server_base.cc:1034] running on GCE node
I20260929 02:58:20.343133 12071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:20.344491 12071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:20.345760 12071 hybrid_clock.cc:648] HybridClock initialized: now 1790650700345722 us; error 49 us; skew 500 ppm
I20260929 02:58:20.346097 12071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:20 dist-test-slave-m1rv krb5kdc[12058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650700, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:20.349164 12071 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:58:20.350664 12071 webserver.cc:460] Webserver started at http://127.8.208.126:43129/ using document root <none> and password file <none>
I20260929 02:58:20.351320 12071 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:20.351460 12071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:20.351747 12071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:20.354628 12071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "2c228681011e45ecb6f45cffcae1a123"
format_stamp: "Formatted at 2026-09-29 02:58:20 on dist-test-slave-m1rv"
server_key: "de8add4806ad1f71401b9cf3caf7132b"
server_key_iv: "5b3c5128789aaf7b2de9fa72fa93589a"
server_key_version: "encryptionkey@0"
I20260929 02:58:20.355358 12071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "2c228681011e45ecb6f45cffcae1a123"
format_stamp: "Formatted at 2026-09-29 02:58:20 on dist-test-slave-m1rv"
server_key: "de8add4806ad1f71401b9cf3caf7132b"
server_key_iv: "5b3c5128789aaf7b2de9fa72fa93589a"
server_key_version: "encryptionkey@0"
I20260929 02:58:20.360312 12071 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:58:20.363610 12086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:20.364851 12071 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:58:20.365020 12071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "2c228681011e45ecb6f45cffcae1a123"
format_stamp: "Formatted at 2026-09-29 02:58:20 on dist-test-slave-m1rv"
server_key: "de8add4806ad1f71401b9cf3caf7132b"
server_key_iv: "5b3c5128789aaf7b2de9fa72fa93589a"
server_key_version: "encryptionkey@0"
I20260929 02:58:20.365238 12071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:20.378185 12071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:20.382380 12071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:20.382650 12071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:20.400689 12071 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:43265
I20260929 02:58:20.400719 12137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:43265 every 8 connection(s)
I20260929 02:58:20.402158 12071 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:20.404400 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 12071
I20260929 02:58:20.404616 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:20.405316 9025 external_mini_cluster.cc:1428] Setting key f4a0f7622c87355b6a31b6d9e0dd3901
I20260929 02:58:20.406664 12138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:58:20 dist-test-slave-m1rv krb5kdc[12058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:20.420310 12138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123: Bootstrap starting.
I20260929 02:58:20.424492 12138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:20.425832 12138 log.cc:824] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:20.428766 12141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:20.407361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38620 (local address 127.8.208.126:43265)
0929 02:58:20.408135 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:20.408151 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:20.408200 (+ 49us) server_negotiation.cc:407] Connection header received
0929 02:58:20.410064 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:20.410091 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:20.410605 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:20.411063 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:20.412243 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:20.414338 (+ 2095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:20.415566 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:20.416600 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:20.416783 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:20.420830 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:20.420840 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:20.420859 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:20.420906 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:20.424755 (+ 3849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:20.425456 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:20.425467 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:20.425482 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:20.425694 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:20.426095 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:20.426101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:20.426110 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:20.426701 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:20.427020 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:20.427458 (+ 438us) server_negotiation.cc:299] Negotiation successful
0929 02:58:20.427839 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":337,"threads_started":1}
I20260929 02:58:20.428870 12138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123: No bootstrap required, opened a new log
I20260929 02:58:20.436892 12138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c228681011e45ecb6f45cffcae1a123" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43265 } }
I20260929 02:58:20.437438 12138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:20.437557 12138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c228681011e45ecb6f45cffcae1a123, State: Initialized, Role: FOLLOWER
I20260929 02:58:20.438112 12138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c228681011e45ecb6f45cffcae1a123" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43265 } }
I20260929 02:58:20.438344 12138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:20.438488 12138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:20.438663 12138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:20.441118 12138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c228681011e45ecb6f45cffcae1a123" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43265 } }
I20260929 02:58:20.441684 12138 leader_election.cc:302] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c228681011e45ecb6f45cffcae1a123; no voters:
I20260929 02:58:20.442273 12138 leader_election.cc:288] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:20.442416 12143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:20.443085 12143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [term 1 LEADER]: Becoming Leader. State: Replica: 2c228681011e45ecb6f45cffcae1a123, State: Running, Role: LEADER
I20260929 02:58:20.443635 12143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c228681011e45ecb6f45cffcae1a123" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43265 } }
I20260929 02:58:20.444375 12138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:20.446698 12144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c228681011e45ecb6f45cffcae1a123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c228681011e45ecb6f45cffcae1a123" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43265 } } }
I20260929 02:58:20.447078 12144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:20.446688 12145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c228681011e45ecb6f45cffcae1a123. Latest consensus state: current_term: 1 leader_uuid: "2c228681011e45ecb6f45cffcae1a123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c228681011e45ecb6f45cffcae1a123" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43265 } } }
I20260929 02:58:20.447726 12145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:20.449922 12151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:20.454461 12151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:20.462728 12151 catalog_manager.cc:1347] Generated new cluster ID: 81f9a6c3e4d1422fbc638dfae2dd171e
I20260929 02:58:20.462847 12151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:20.472234 12151 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:20.473606 12151 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:20.484760 12151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c228681011e45ecb6f45cffcae1a123: Generated new TSK 0
I20260929 02:58:20.485656 12151 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 02:58:20.529117 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:43265
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:20.853437 12164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:20.853714 12164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:20.853902 12164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:20.861439 12164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:20.861582 12164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:20.861761 12164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:58:20.872025 12164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:43265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:20.873418 12164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:20.874720 12164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:20.887841 12169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:20.887931 12170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:20.887872 12172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:20.890555 12164 server_base.cc:1034] running on GCE node
I20260929 02:58:20.891140 12164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:20.892033 12164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:20.893244 12164 hybrid_clock.cc:648] HybridClock initialized: now 1790650700893196 us; error 33 us; skew 500 ppm
I20260929 02:58:20.893550 12164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:20 dist-test-slave-m1rv krb5kdc[12058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650700, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:20.896816 12164 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 02:58:20.898442 12164 webserver.cc:460] Webserver started at http://127.8.208.65:43489/ using document root <none> and password file <none>
I20260929 02:58:20.899134 12164 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:20.899258 12164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:20.899569 12164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:20.902412 12164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "02146182c06d46f8aecf13f37544553e"
format_stamp: "Formatted at 2026-09-29 02:58:20 on dist-test-slave-m1rv"
server_key: "2edff66d081d429da35272c69c062196"
server_key_iv: "c584c0c8af9977d806032bda72ca62eb"
server_key_version: "encryptionkey@0"
I20260929 02:58:20.903177 12164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "02146182c06d46f8aecf13f37544553e"
format_stamp: "Formatted at 2026-09-29 02:58:20 on dist-test-slave-m1rv"
server_key: "2edff66d081d429da35272c69c062196"
server_key_iv: "c584c0c8af9977d806032bda72ca62eb"
server_key_version: "encryptionkey@0"
I20260929 02:58:20.908178 12164 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260929 02:58:20.911481 12179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:20.912772 12164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:20.912948 12164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "02146182c06d46f8aecf13f37544553e"
format_stamp: "Formatted at 2026-09-29 02:58:20 on dist-test-slave-m1rv"
server_key: "2edff66d081d429da35272c69c062196"
server_key_iv: "c584c0c8af9977d806032bda72ca62eb"
server_key_version: "encryptionkey@0"
I20260929 02:58:20.913138 12164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:20.934243 12164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:20.938505 12164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:20.938787 12164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:20.939697 12164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:20.940977 12164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:20.941077 12164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:20.941242 12164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:20.941318 12164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260929 02:58:20.973699 12164 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:43909
I20260929 02:58:20.973757 12291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:43909 every 8 connection(s)
I20260929 02:58:20.975010 12164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:58:20.977869 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 12164
I20260929 02:58:20.978058 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:58:20.978914 9025 external_mini_cluster.cc:1428] Setting key 04f5dc47223768b7897858ecb62c0bbc
Sep 29 02:58:20 dist-test-slave-m1rv krb5kdc[12058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650700, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
I20260929 02:58:20.996136 12294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:20.977587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:37855 (local address 127.8.208.126:43265)
0929 02:58:20.978141 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:20.978148 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:20.979305 (+ 1157us) server_negotiation.cc:407] Connection header received
0929 02:58:20.981682 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:20.981687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:20.981775 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:20.981890 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:20.983381 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:20.984294 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:20.986104 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:20.986513 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:20.986650 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:20.990467 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:20.990471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:20.990474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:20.990504 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:20.993035 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:20.993817 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:20.993829 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:20.993831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:20.993950 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:20.994388 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:20.994392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:20.994396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:20.994685 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:20.994838 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:20.995551 (+ 713us) server_negotiation.cc:299] Negotiation successful
0929 02:58:20.995769 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":321,"threads_started":1}
I20260929 02:58:20.997291 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:20.978314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:43265 (local address 127.8.208.65:37855)
0929 02:58:20.979146 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:20.979190 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:20.981426 (+ 2236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:20.982140 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:20.982149 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:20.982662 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:20.983213 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:20.983229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:20.984458 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:20.984464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:20.985953 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:20.985965 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:20.986740 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:20.986747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:20.987149 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:20.987994 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:20.988021 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:20.990226 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:20.993211 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:20.993221 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:20.993238 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:20.993685 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:20.994063 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:20.994068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:20.994072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:20.994266 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:20.994827 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:20.994835 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:20.995185 (+ 350us) client_negotiation.cc:769] Sending connection context
0929 02:58:20.995452 (+ 267us) client_negotiation.cc:240] Negotiation successful
0929 02:58:20.996014 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":303,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
I20260929 02:58:20.999185 12292 heartbeater.cc:347] Connected to a master server at 127.8.208.126:43265
I20260929 02:58:20.999500 12292 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:21.000238 12292 heartbeater.cc:511] Master 127.8.208.126:43265 requested a full tablet report, sending...
I20260929 02:58:21.002665 12103 ts_manager.cc:194] Registered new tserver with Master: 02146182c06d46f8aecf13f37544553e (127.8.208.65:43909)
I20260929 02:58:21.004331 12103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:37855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 02:58:21.023964 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:43265
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:21.357210 12298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:21.357685 12298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:21.357941 12298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:21.365456 12298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:21.365553 12298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:21.365756 12298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:58:21.375463 12298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:43265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:21.376729 12298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:21.378022 12298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:21.390713 12303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:21.390669 12306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:21.390746 12304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:21.391804 12298 server_base.cc:1034] running on GCE node
I20260929 02:58:21.392323 12298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:21.393128 12298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:21.394368 12298 hybrid_clock.cc:648] HybridClock initialized: now 1790650701394320 us; error 39 us; skew 500 ppm
I20260929 02:58:21.394675 12298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:21 dist-test-slave-m1rv krb5kdc[12058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650701, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:21.398008 12298 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 02:58:21.399418 12298 webserver.cc:460] Webserver started at http://127.8.208.66:37043/ using document root <none> and password file <none>
I20260929 02:58:21.400171 12298 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:21.400321 12298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:21.400785 12298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:21.404748 12298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "6a698fd661d240f7b2be9f6bd1d06da6"
format_stamp: "Formatted at 2026-09-29 02:58:21 on dist-test-slave-m1rv"
server_key: "ad5234f6360fcb65d3cb5af1a20ead66"
server_key_iv: "9e10be1ae28311359c17cac98039060a"
server_key_version: "encryptionkey@0"
I20260929 02:58:21.405867 12298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "6a698fd661d240f7b2be9f6bd1d06da6"
format_stamp: "Formatted at 2026-09-29 02:58:21 on dist-test-slave-m1rv"
server_key: "ad5234f6360fcb65d3cb5af1a20ead66"
server_key_iv: "9e10be1ae28311359c17cac98039060a"
server_key_version: "encryptionkey@0"
I20260929 02:58:21.412513 12298 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260929 02:58:21.417030 12313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:21.418527 12298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260929 02:58:21.418680 12298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "6a698fd661d240f7b2be9f6bd1d06da6"
format_stamp: "Formatted at 2026-09-29 02:58:21 on dist-test-slave-m1rv"
server_key: "ad5234f6360fcb65d3cb5af1a20ead66"
server_key_iv: "9e10be1ae28311359c17cac98039060a"
server_key_version: "encryptionkey@0"
I20260929 02:58:21.418855 12298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:21.432619 12298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:21.436362 12298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:21.436708 12298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:21.437683 12298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:21.438933 12298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:21.439030 12298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:21.439167 12298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:21.439240 12298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:21.470899 12298 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:38427
I20260929 02:58:21.470954 12425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:38427 every 8 connection(s)
I20260929 02:58:21.472208 12298 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:58:21.482062 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 12298
I20260929 02:58:21.482236 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:58:21.482789 9025 external_mini_cluster.cc:1428] Setting key 87781edc1c25e14ff9e170db8824874c
Sep 29 02:58:21 dist-test-slave-m1rv krb5kdc[12058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650701, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:21.492792 12294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:21.474765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:34175 (local address 127.8.208.126:43265)
0929 02:58:21.474941 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:21.474946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:21.476120 (+ 1174us) server_negotiation.cc:407] Connection header received
0929 02:58:21.477957 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:21.477962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:21.478053 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:21.478161 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:21.479593 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:21.480308 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:21.481876 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:21.482389 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:21.482519 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:21.486549 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:21.486555 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:21.486558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:21.486587 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:21.489225 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:21.490259 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:21.490267 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:21.490272 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:21.490458 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:21.490869 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:21.490873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:21.490877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:21.491261 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:21.491425 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:21.492068 (+ 643us) server_negotiation.cc:299] Negotiation successful
0929 02:58:21.492343 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:58:21.493521 12428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:21.475281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:43265 (local address 127.8.208.66:34175)
0929 02:58:21.475970 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:21.476007 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:21.477696 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:21.478389 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:21.478397 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:21.478888 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:21.479421 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:21.479437 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:21.480456 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:21.480463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:21.481745 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:21.481755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:21.482598 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:21.482605 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:21.482827 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:21.483568 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:21.483598 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:21.486325 (+ 2727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:21.489371 (+ 3046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:21.489383 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:21.489412 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:21.490132 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:21.490568 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:21.490572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:21.490574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:21.490755 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:21.491378 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:21.491384 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:21.491706 (+ 322us) client_negotiation.cc:769] Sending connection context
0929 02:58:21.491925 (+ 219us) client_negotiation.cc:240] Negotiation successful
0929 02:58:21.492320 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":266,"threads_started":1}
I20260929 02:58:21.495054 12426 heartbeater.cc:347] Connected to a master server at 127.8.208.126:43265
I20260929 02:58:21.495368 12426 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:21.496084 12426 heartbeater.cc:511] Master 127.8.208.126:43265 requested a full tablet report, sending...
I20260929 02:58:21.497690 12103 ts_manager.cc:194] Registered new tserver with Master: 6a698fd661d240f7b2be9f6bd1d06da6 (127.8.208.66:38427)
I20260929 02:58:21.499029 12103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:34175
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 02:58:21.526443 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:43265
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:21.841138 12431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:21.841441 12431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:21.841650 12431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:21.848903 12431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:21.849031 12431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:21.849247 12431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:58:21.858574 12431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:43265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:21.859874 12431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:21.861186 12431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:21.874886 12437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:21.874923 12436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:21.875931 12439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:21.877147 12431 server_base.cc:1034] running on GCE node
I20260929 02:58:21.877938 12431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:21.879040 12431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:21.880611 12431 hybrid_clock.cc:648] HybridClock initialized: now 1790650701880520 us; error 57 us; skew 500 ppm
I20260929 02:58:21.880944 12431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:21 dist-test-slave-m1rv krb5kdc[12058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650701, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:21.884084 12431 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 02:58:21.885555 12431 webserver.cc:460] Webserver started at http://127.8.208.67:45797/ using document root <none> and password file <none>
I20260929 02:58:21.886201 12431 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:21.886372 12431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:21.886655 12431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:21.889673 12431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "140d6c98de7e48cba1eebbeea3a0461f"
format_stamp: "Formatted at 2026-09-29 02:58:21 on dist-test-slave-m1rv"
server_key: "1e18b534526d96f5d14c1c81c4a465b9"
server_key_iv: "0cca634ecc318b84692e5cb38c1acee0"
server_key_version: "encryptionkey@0"
I20260929 02:58:21.890420 12431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "140d6c98de7e48cba1eebbeea3a0461f"
format_stamp: "Formatted at 2026-09-29 02:58:21 on dist-test-slave-m1rv"
server_key: "1e18b534526d96f5d14c1c81c4a465b9"
server_key_iv: "0cca634ecc318b84692e5cb38c1acee0"
server_key_version: "encryptionkey@0"
I20260929 02:58:21.895464 12431 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:58:21.898761 12446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:21.900478 12431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:58:21.900651 12431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "140d6c98de7e48cba1eebbeea3a0461f"
format_stamp: "Formatted at 2026-09-29 02:58:21 on dist-test-slave-m1rv"
server_key: "1e18b534526d96f5d14c1c81c4a465b9"
server_key_iv: "0cca634ecc318b84692e5cb38c1acee0"
server_key_version: "encryptionkey@0"
I20260929 02:58:21.900843 12431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:21.976315 12431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:21.980700 12431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:21.981025 12431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:21.981995 12431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:21.983289 12431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:21.983388 12431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:21.983525 12431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:21.983593 12431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:22.008054 12292 heartbeater.cc:503] Master 127.8.208.126:43265 was elected leader, sending a full tablet report...
I20260929 02:58:22.017036 12431 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:44385
I20260929 02:58:22.017082 12558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:44385 every 8 connection(s)
I20260929 02:58:22.018363 12431 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:58:22.025008 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 12431
I20260929 02:58:22.025219 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:58:22.025655 9025 external_mini_cluster.cc:1428] Setting key 34329f1e7847bcdffb6636abee8e4f93
Sep 29 02:58:22 dist-test-slave-m1rv krb5kdc[12058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650701, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:22.037849 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:22.020794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:46313 (local address 127.8.208.126:43265)
0929 02:58:22.021282 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:22.021286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:22.022249 (+ 963us) server_negotiation.cc:407] Connection header received
0929 02:58:22.024161 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:22.024166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:22.024261 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:22.024384 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:22.025992 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:22.027127 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:22.028842 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:22.029301 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:22.029441 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:22.032532 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:22.032546 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:22.032550 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:22.032578 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:22.034954 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:22.035632 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:22.035640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:22.035644 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:22.035768 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:22.036184 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:22.036191 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:22.036194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:22.036467 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:22.036620 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:22.037270 (+ 650us) server_negotiation.cc:299] Negotiation successful
0929 02:58:22.037487 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":242,"threads_started":1}
I20260929 02:58:22.038715 12562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:22.021418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:43265 (local address 127.8.208.67:46313)
0929 02:58:22.022096 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:22.022133 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:22.023904 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:22.024609 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:22.024617 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:22.025160 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:22.025827 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:22.025846 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:22.027260 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:22.027266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:22.028703 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:22.028714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:22.029520 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:22.029529 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:22.029750 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:22.030333 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:22.030353 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:22.032341 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:22.035086 (+ 2745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:22.035095 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:22.035109 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:22.035516 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:22.035893 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:22.035898 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:22.035902 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:22.036075 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:22.036597 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:22.036605 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:22.036911 (+ 306us) client_negotiation.cc:769] Sending connection context
0929 02:58:22.037178 (+ 267us) client_negotiation.cc:240] Negotiation successful
0929 02:58:22.037689 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":271,"threads_started":1}
I20260929 02:58:22.040180 12559 heartbeater.cc:347] Connected to a master server at 127.8.208.126:43265
I20260929 02:58:22.040501 12559 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:22.041267 12559 heartbeater.cc:511] Master 127.8.208.126:43265 requested a full tablet report, sending...
I20260929 02:58:22.042569 12102 ts_manager.cc:194] Registered new tserver with Master: 140d6c98de7e48cba1eebbeea3a0461f (127.8.208.67:44385)
I20260929 02:58:22.043388 12102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:46313
I20260929 02:58:22.052788 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:58:22 dist-test-slave-m1rv krb5kdc[12058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650702, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 02:58:22 dist-test-slave-m1rv krb5kdc[12058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650702, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:22.097463 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:22.084609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38626 (local address 127.8.208.126:43265)
0929 02:58:22.084797 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:22.084802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:22.085318 (+ 516us) server_negotiation.cc:407] Connection header received
0929 02:58:22.085478 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:22.085482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:22.085546 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:22.085635 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:22.086975 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:22.087678 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:22.088825 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:22.089273 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:22.089377 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:22.093023 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:22.093028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:22.093031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:22.093061 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:22.095385 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:22.095926 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:22.095930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:22.095932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:22.096035 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:22.096326 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:22.096329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:22.096331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:22.096639 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:22.096805 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:22.097030 (+ 225us) server_negotiation.cc:299] Negotiation successful
0929 02:58:22.097235 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260929 02:58:22.098018 12102 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38626
I20260929 02:58:22.099498 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 12164
Sep 29 02:58:22 dist-test-slave-m1rv krb5kdc[12058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650702, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:58:22.114462 12571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:22.100669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.8.208.65:43909)
0929 02:58:22.101303 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:22.101311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:22.101334 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:58:22.101393 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:22.101398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:22.101608 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:22.101740 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:22.102781 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:22.103735 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:22.104857 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:22.105585 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:22.105678 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:22.109396 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:22.109406 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:22.109421 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:22.109450 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:22.112178 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:22.112774 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:22.112781 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:22.112783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:22.112893 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:22.113246 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:22.113249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:22.113251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:22.113555 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:22.113766 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:22.113998 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 02:58:22.114176 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":307,"threads_started":1}
W20260929 02:58:22.115662 12287 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57744
W20260929 02:58:22.116786 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 12164: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:58:22.116993 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 12164
I20260929 02:58:22.132709 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 12298
Sep 29 02:58:22 dist-test-slave-m1rv krb5kdc[12058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650702, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 02:58:22.148101 12572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:22.133616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41272 (local address 127.8.208.66:38427)
0929 02:58:22.134149 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:22.134164 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:22.134189 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:58:22.134248 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:22.134254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:22.134467 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:22.134590 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:22.135641 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:22.136681 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:22.138012 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:22.138552 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:22.138652 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:22.142415 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:22.142427 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:22.142441 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:22.142471 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:22.145496 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:22.146121 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:22.146128 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:22.146130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:22.146243 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:22.146592 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:22.146598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:22.146602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:22.146917 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:22.147219 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:22.147516 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 02:58:22.147777 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":283,"threads_started":1}
W20260929 02:58:22.149550 12421 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41272
W20260929 02:58:22.150665 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 12298: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:58:22.150779 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 12298
I20260929 02:58:22.165767 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 12431
Sep 29 02:58:22 dist-test-slave-m1rv krb5kdc[12058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650702, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 02:58:22.181959 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:22.166654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43734 (local address 127.8.208.67:44385)
0929 02:58:22.167193 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:22.167200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:22.167227 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:58:22.167284 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:22.167291 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:22.167498 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:22.167628 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:22.168891 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:22.170144 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:22.171327 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:22.171903 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:22.172000 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:22.176467 (+ 4467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:22.176478 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:22.176493 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:22.176531 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:22.179492 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:22.180089 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:22.180096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:22.180098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:22.180222 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:22.180523 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:22.180526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:22.180528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:22.180846 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:22.181085 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:22.181340 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 02:58:22.181549 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":268,"threads_started":1}
W20260929 02:58:22.183213 12554 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43734
W20260929 02:58:22.184402 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 12431: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:58:22.184592 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 12431
I20260929 02:58:22.199440 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 12071
I20260929 02:58:22.200497 12133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:22.303645 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 12071
2026-09-29T02:58:22Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3578 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:58:22 dist-test-slave-m1rv krb5kdc[12577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:58:22 dist-test-slave-m1rv krb5kdc[12577](info): set up 2 sockets
Sep 29 02:58:22 dist-test-slave-m1rv krb5kdc[12577](info): commencing operation
krb5kdc: starting...
W20260929 02:58:23.393594 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:58:23 dist-test-slave-m1rv krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:58:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:58:23.588514 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:35237
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:35237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:23.913008 12590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:23.913313 12590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:23.913486 12590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:23.921100 12590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:23.921290 12590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:23.921373 12590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:23.921455 12590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:23.921542 12590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:23.931358 12590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:35237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:35237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:23.932654 12590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:23.934093 12590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:23.946321 12595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:23.946367 12596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:23.946338 12598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:23.946841 12590 server_base.cc:1034] running on GCE node
I20260929 02:58:23.947757 12590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:23.949332 12590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:23.950574 12590 hybrid_clock.cc:648] HybridClock initialized: now 1790650703950517 us; error 50 us; skew 500 ppm
I20260929 02:58:23.950917 12590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:23 dist-test-slave-m1rv krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650703, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:23.959442 12590 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:58:23.960891 12590 webserver.cc:460] Webserver started at http://127.8.208.126:37661/ using document root <none> and password file <none>
I20260929 02:58:23.961649 12590 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:23.961792 12590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:23.962106 12590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:23.965085 12590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "dd187cb2b6ce4d85a13209422b391c91"
format_stamp: "Formatted at 2026-09-29 02:58:23 on dist-test-slave-m1rv"
server_key: "22f1ac48a9f6d91187734bcaa078fd05"
server_key_iv: "6a6eb86a6227be242fe3688192c7406f"
server_key_version: "encryptionkey@0"
I20260929 02:58:23.966008 12590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "dd187cb2b6ce4d85a13209422b391c91"
format_stamp: "Formatted at 2026-09-29 02:58:23 on dist-test-slave-m1rv"
server_key: "22f1ac48a9f6d91187734bcaa078fd05"
server_key_iv: "6a6eb86a6227be242fe3688192c7406f"
server_key_version: "encryptionkey@0"
I20260929 02:58:23.971055 12590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:58:23.974678 12605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:23.976173 12590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:58:23.976351 12590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "dd187cb2b6ce4d85a13209422b391c91"
format_stamp: "Formatted at 2026-09-29 02:58:23 on dist-test-slave-m1rv"
server_key: "22f1ac48a9f6d91187734bcaa078fd05"
server_key_iv: "6a6eb86a6227be242fe3688192c7406f"
server_key_version: "encryptionkey@0"
I20260929 02:58:23.976572 12590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:23.989904 12590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:23.993921 12590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:23.994213 12590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:24.013425 12590 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:35237
I20260929 02:58:24.013451 12656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:35237 every 8 connection(s)
I20260929 02:58:24.014886 12590 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:24.016568 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 12590
I20260929 02:58:24.016805 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:24.017433 9025 external_mini_cluster.cc:1428] Setting key 08db866283dcf33bad5961e08a52d72f
I20260929 02:58:24.019359 12657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:58:24 dist-test-slave-m1rv krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:24.032958 12657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91: Bootstrap starting.
I20260929 02:58:24.036381 12657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:24.037485 12657 log.cc:824] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:24.040241 12660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:24.019336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55122 (local address 127.8.208.126:35237)
0929 02:58:24.020168 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:24.020180 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:24.020218 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:58:24.021905 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:24.021927 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:24.022311 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:24.022650 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:24.024435 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:24.025866 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:24.027619 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:24.028261 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:24.028409 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:24.033054 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:24.033062 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:24.033076 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:24.033146 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:24.036268 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:24.037010 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:24.037019 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:24.037034 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:24.037258 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:24.037637 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:24.037645 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:24.037648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:24.038163 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:24.038463 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:24.038915 (+ 452us) server_negotiation.cc:299] Negotiation successful
0929 02:58:24.039308 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":366,"threads_started":1}
I20260929 02:58:24.040390 12657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91: No bootstrap required, opened a new log
I20260929 02:58:24.048058 12657 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd187cb2b6ce4d85a13209422b391c91" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35237 } }
I20260929 02:58:24.048465 12657 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:24.048590 12657 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd187cb2b6ce4d85a13209422b391c91, State: Initialized, Role: FOLLOWER
I20260929 02:58:24.049247 12657 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd187cb2b6ce4d85a13209422b391c91" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35237 } }
I20260929 02:58:24.049504 12657 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:24.049643 12657 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:24.049815 12657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:24.051900 12657 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd187cb2b6ce4d85a13209422b391c91" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35237 } }
I20260929 02:58:24.052347 12657 leader_election.cc:302] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd187cb2b6ce4d85a13209422b391c91; no voters:
I20260929 02:58:24.052910 12657 leader_election.cc:288] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:24.053004 12662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:24.053763 12662 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [term 1 LEADER]: Becoming Leader. State: Replica: dd187cb2b6ce4d85a13209422b391c91, State: Running, Role: LEADER
I20260929 02:58:24.054345 12662 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd187cb2b6ce4d85a13209422b391c91" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35237 } }
I20260929 02:58:24.055317 12657 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:24.057528 12664 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd187cb2b6ce4d85a13209422b391c91. Latest consensus state: current_term: 1 leader_uuid: "dd187cb2b6ce4d85a13209422b391c91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd187cb2b6ce4d85a13209422b391c91" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35237 } } }
I20260929 02:58:24.057930 12664 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:24.057514 12663 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd187cb2b6ce4d85a13209422b391c91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd187cb2b6ce4d85a13209422b391c91" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35237 } } }
I20260929 02:58:24.058188 12663 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:24.062263 12668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:24.067839 12668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:24.076895 12668 catalog_manager.cc:1347] Generated new cluster ID: 666f7903fa274455852c55ad13e6c107
I20260929 02:58:24.077032 12668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:24.093914 12668 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:24.095287 12668 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:24.101892 12668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd187cb2b6ce4d85a13209422b391c91: Generated new TSK 0
I20260929 02:58:24.102690 12668 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 02:58:24.150815 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35237
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:24.469475 12683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:24.469740 12683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:24.469831 12683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:24.476780 12683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:24.476887 12683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:24.476999 12683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:58:24.486662 12683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:24.487923 12683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:24.489138 12683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:24.501966 12691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:24.502031 12688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:24.502007 12689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:24.502769 12683 server_base.cc:1034] running on GCE node
I20260929 02:58:24.503312 12683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:24.504264 12683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:24.505455 12683 hybrid_clock.cc:648] HybridClock initialized: now 1790650704505408 us; error 51 us; skew 500 ppm
I20260929 02:58:24.505736 12683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:24 dist-test-slave-m1rv krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:24.508738 12683 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 02:58:24.510120 12683 webserver.cc:460] Webserver started at http://127.8.208.65:40587/ using document root <none> and password file <none>
I20260929 02:58:24.510790 12683 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:24.510888 12683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:24.511147 12683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:24.514030 12683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "ca9fb5a999494d80bb7c90f44b7a12a5"
format_stamp: "Formatted at 2026-09-29 02:58:24 on dist-test-slave-m1rv"
server_key: "47cab90f7be9e156e7aa6165fbe887db"
server_key_iv: "fe82cd88e388b60ce56e3bf530341c31"
server_key_version: "encryptionkey@0"
I20260929 02:58:24.514739 12683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "ca9fb5a999494d80bb7c90f44b7a12a5"
format_stamp: "Formatted at 2026-09-29 02:58:24 on dist-test-slave-m1rv"
server_key: "47cab90f7be9e156e7aa6165fbe887db"
server_key_iv: "fe82cd88e388b60ce56e3bf530341c31"
server_key_version: "encryptionkey@0"
I20260929 02:58:24.519822 12683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:58:24.523025 12698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:24.524273 12683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:24.524410 12683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "ca9fb5a999494d80bb7c90f44b7a12a5"
format_stamp: "Formatted at 2026-09-29 02:58:24 on dist-test-slave-m1rv"
server_key: "47cab90f7be9e156e7aa6165fbe887db"
server_key_iv: "fe82cd88e388b60ce56e3bf530341c31"
server_key_version: "encryptionkey@0"
I20260929 02:58:24.524569 12683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:24.548280 12683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:24.552232 12683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:24.552472 12683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:24.553401 12683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:24.554636 12683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:24.554708 12683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:24.554787 12683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:24.554829 12683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:24.588948 12683 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:34147
I20260929 02:58:24.589063 12810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:34147 every 8 connection(s)
I20260929 02:58:24.590281 12683 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:58:24.599553 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 12683
I20260929 02:58:24.599783 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:58:24.600411 9025 external_mini_cluster.cc:1428] Setting key 6de0932551c3cb7ccd804b4fd1c2adf1
Sep 29 02:58:24 dist-test-slave-m1rv krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:24.609967 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:24.592703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:36307 (local address 127.8.208.126:35237)
0929 02:58:24.593127 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:24.593132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:24.594111 (+ 979us) server_negotiation.cc:407] Connection header received
0929 02:58:24.595943 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:24.595949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:24.596038 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:24.596149 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:24.597648 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:24.598371 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:24.599947 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:24.600357 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:24.600493 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:24.604414 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:24.604418 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:24.604421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:24.604450 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:24.606964 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:24.607675 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:24.607687 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:24.607689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:24.607811 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:24.608249 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:24.608254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:24.608257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:24.608529 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:24.608698 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:24.609404 (+ 706us) server_negotiation.cc:299] Negotiation successful
0929 02:58:24.609616 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":225,"threads_started":1}
I20260929 02:58:24.610960 12814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:24.593289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35237 (local address 127.8.208.65:36307)
0929 02:58:24.593955 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:24.593998 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:24.595678 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:24.596385 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:24.596393 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:24.596882 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:24.597487 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:24.597505 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:24.598523 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:24.598532 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:24.599797 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:24.599809 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:24.600572 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:24.600581 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:24.600829 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:24.601633 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:24.601664 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:24.604189 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:24.607110 (+ 2921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:24.607119 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:24.607136 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:24.607548 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:24.607932 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:24.607938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:24.607942 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:24.608130 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:24.608661 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:24.608670 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:24.609007 (+ 337us) client_negotiation.cc:769] Sending connection context
0929 02:58:24.609303 (+ 296us) client_negotiation.cc:240] Negotiation successful
0929 02:58:24.609828 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":249,"threads_started":1}
I20260929 02:58:24.612486 12811 heartbeater.cc:347] Connected to a master server at 127.8.208.126:35237
I20260929 02:58:24.612819 12811 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:24.613701 12811 heartbeater.cc:511] Master 127.8.208.126:35237 requested a full tablet report, sending...
I20260929 02:58:24.615789 12622 ts_manager.cc:194] Registered new tserver with Master: ca9fb5a999494d80bb7c90f44b7a12a5 (127.8.208.65:34147)
I20260929 02:58:24.617388 12622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:36307
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 02:58:24.644089 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35237
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:24.960268 12817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:24.960525 12817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:24.960615 12817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:24.967876 12817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:24.967979 12817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:24.968087 12817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:58:24.977700 12817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:24.978888 12817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:24.980208 12817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:24.993281 12823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:24.993237 12825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:24.993322 12822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:24.994069 12817 server_base.cc:1034] running on GCE node
I20260929 02:58:24.994609 12817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:24.995493 12817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:24.996697 12817 hybrid_clock.cc:648] HybridClock initialized: now 1790650704996660 us; error 30 us; skew 500 ppm
I20260929 02:58:24.997010 12817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:24 dist-test-slave-m1rv krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:25.000170 12817 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 02:58:25.001715 12817 webserver.cc:460] Webserver started at http://127.8.208.66:45001/ using document root <none> and password file <none>
I20260929 02:58:25.002403 12817 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:25.002530 12817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:25.002847 12817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:25.005824 12817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "7a200c0293984e26bb46bb0f8c441e2a"
format_stamp: "Formatted at 2026-09-29 02:58:24 on dist-test-slave-m1rv"
server_key: "a523efdb3a611689bda7b4f53c17a925"
server_key_iv: "94b51502ef00d3b5c0d8c89250a645cb"
server_key_version: "encryptionkey@0"
I20260929 02:58:25.006677 12817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "7a200c0293984e26bb46bb0f8c441e2a"
format_stamp: "Formatted at 2026-09-29 02:58:24 on dist-test-slave-m1rv"
server_key: "a523efdb3a611689bda7b4f53c17a925"
server_key_iv: "94b51502ef00d3b5c0d8c89250a645cb"
server_key_version: "encryptionkey@0"
I20260929 02:58:25.011971 12817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:58:25.015614 12832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:25.017001 12817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:25.017284 12817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "7a200c0293984e26bb46bb0f8c441e2a"
format_stamp: "Formatted at 2026-09-29 02:58:24 on dist-test-slave-m1rv"
server_key: "a523efdb3a611689bda7b4f53c17a925"
server_key_iv: "94b51502ef00d3b5c0d8c89250a645cb"
server_key_version: "encryptionkey@0"
I20260929 02:58:25.017515 12817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:25.047871 12817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:25.051812 12817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:25.052109 12817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:25.053031 12817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:25.054337 12817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:25.054447 12817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:25.054591 12817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:25.054669 12817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:25.086858 12817 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:43401
I20260929 02:58:25.086912 12944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:43401 every 8 connection(s)
I20260929 02:58:25.088160 12817 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:58:25.091651 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 12817
I20260929 02:58:25.091881 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:58:25.092365 9025 external_mini_cluster.cc:1428] Setting key 8f09c5f1104b3ca3978d9edf163d830f
Sep 29 02:58:25 dist-test-slave-m1rv krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650704, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:25.109489 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:25.090690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:39341 (local address 127.8.208.126:35237)
0929 02:58:25.090898 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:25.090905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:25.092230 (+ 1325us) server_negotiation.cc:407] Connection header received
0929 02:58:25.094313 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:25.094316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:25.094377 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:25.094492 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:25.095926 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.096711 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.099292 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.100011 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.100153 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:25.103880 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:25.103885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:25.103889 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:25.103920 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:25.106512 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.107253 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.107257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.107259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.107385 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.107817 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.107827 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.107829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.108075 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:25.108229 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:25.108890 (+ 661us) server_negotiation.cc:299] Negotiation successful
0929 02:58:25.109115 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
I20260929 02:58:25.110507 12947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:25.091231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35237 (local address 127.8.208.66:39341)
0929 02:58:25.092074 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:25.092117 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:25.094032 (+ 1915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:25.094728 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:25.094738 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:25.095207 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:25.095755 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:25.095772 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.096895 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.096903 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:25.099148 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:25.099167 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.100220 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.100224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:25.100427 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:25.101119 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:25.101141 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:25.103658 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:25.106666 (+ 3008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:25.106675 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:25.106691 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:25.107106 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:25.107504 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:25.107509 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:25.107513 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:25.107696 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:25.108203 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:25.108210 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:25.108546 (+ 336us) client_negotiation.cc:769] Sending connection context
0929 02:58:25.108798 (+ 252us) client_negotiation.cc:240] Negotiation successful
0929 02:58:25.109361 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":328,"threads_started":1}
I20260929 02:58:25.112113 12945 heartbeater.cc:347] Connected to a master server at 127.8.208.126:35237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
I20260929 02:58:25.112417 12945 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:25.113123 12945 heartbeater.cc:511] Master 127.8.208.126:35237 requested a full tablet report, sending...
I20260929 02:58:25.114583 12622 ts_manager.cc:194] Registered new tserver with Master: 7a200c0293984e26bb46bb0f8c441e2a (127.8.208.66:43401)
I20260929 02:58:25.115365 12622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:39341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 02:58:25.134589 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35237
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:25.446378 12950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:25.446650 12950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:25.446846 12950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:25.454649 12950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:25.454778 12950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:25.454957 12950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:58:25.464957 12950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:25.466265 12950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:25.467545 12950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:25.480676 12956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:25.480832 12955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:25.480703 12958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:25.481532 12950 server_base.cc:1034] running on GCE node
I20260929 02:58:25.482136 12950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:25.483078 12950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:25.484301 12950 hybrid_clock.cc:648] HybridClock initialized: now 1790650705484245 us; error 53 us; skew 500 ppm
I20260929 02:58:25.484612 12950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:25 dist-test-slave-m1rv krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:25.487799 12950 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 02:58:25.489110 12950 webserver.cc:460] Webserver started at http://127.8.208.67:36299/ using document root <none> and password file <none>
I20260929 02:58:25.489848 12950 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:25.489985 12950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:25.490291 12950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:25.493211 12950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "41684487297c40f0a08335ffc45c9208"
format_stamp: "Formatted at 2026-09-29 02:58:25 on dist-test-slave-m1rv"
server_key: "dc691b4d8e508812e528504662665bee"
server_key_iv: "b4ddfc6a8df9ea637aba194badc75f1c"
server_key_version: "encryptionkey@0"
I20260929 02:58:25.494000 12950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "41684487297c40f0a08335ffc45c9208"
format_stamp: "Formatted at 2026-09-29 02:58:25 on dist-test-slave-m1rv"
server_key: "dc691b4d8e508812e528504662665bee"
server_key_iv: "b4ddfc6a8df9ea637aba194badc75f1c"
server_key_version: "encryptionkey@0"
I20260929 02:58:25.499207 12950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:58:25.502666 12965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:25.503968 12950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:58:25.504142 12950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "41684487297c40f0a08335ffc45c9208"
format_stamp: "Formatted at 2026-09-29 02:58:25 on dist-test-slave-m1rv"
server_key: "dc691b4d8e508812e528504662665bee"
server_key_iv: "b4ddfc6a8df9ea637aba194badc75f1c"
server_key_version: "encryptionkey@0"
I20260929 02:58:25.504364 12950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:25.529390 12950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:25.533601 12950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:25.533933 12950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:25.534888 12950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:25.536157 12950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:25.536262 12950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:25.536432 12950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:25.536513 12950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:25.571403 12950 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:34261
I20260929 02:58:25.571480 13077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:34261 every 8 connection(s)
I20260929 02:58:25.572682 12950 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:58:25.581919 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 12950
I20260929 02:58:25.582130 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:58:25.582527 9025 external_mini_cluster.cc:1428] Setting key f6433167a47aa238cf027a6c484c71c4
Sep 29 02:58:25 dist-test-slave-m1rv krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650705, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:25.592310 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:25.575165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:59247 (local address 127.8.208.126:35237)
0929 02:58:25.575328 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:25.575332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:25.576497 (+ 1165us) server_negotiation.cc:407] Connection header received
0929 02:58:25.578378 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:25.578383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:25.578506 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:25.578622 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:25.580060 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.580819 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.582481 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.582950 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.583101 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:25.586678 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:25.586683 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:25.586686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:25.586717 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:25.589387 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.590090 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.590095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.590097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.590214 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.590642 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.590647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.590650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.590930 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:25.591081 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:25.591729 (+ 648us) server_negotiation.cc:299] Negotiation successful
0929 02:58:25.591963 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:58:25.593259 13080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:25.575652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35237 (local address 127.8.208.67:59247)
0929 02:58:25.576334 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:25.576371 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:25.578115 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:25.578851 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:25.578859 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:25.579351 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:25.579885 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:25.579902 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.580988 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.580991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:25.582342 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:25.582354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.583220 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.583229 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:25.583533 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:25.584224 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:25.584245 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:25.586488 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:25.589532 (+ 3044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:25.589541 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:25.589555 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:25.589960 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:25.590324 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:25.590329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:25.590333 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:25.590517 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:25.591071 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:25.591079 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:25.591376 (+ 297us) client_negotiation.cc:769] Sending connection context
0929 02:58:25.591623 (+ 247us) client_negotiation.cc:240] Negotiation successful
0929 02:58:25.592145 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":276,"threads_started":1}
I20260929 02:58:25.594483 13078 heartbeater.cc:347] Connected to a master server at 127.8.208.126:35237
I20260929 02:58:25.594733 13078 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:25.595333 13078 heartbeater.cc:511] Master 127.8.208.126:35237 requested a full tablet report, sending...
I20260929 02:58:25.596686 12622 ts_manager.cc:194] Registered new tserver with Master: 41684487297c40f0a08335ffc45c9208 (127.8.208.67:34261)
I20260929 02:58:25.597746 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:58:25.597756 12622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:59247
I20260929 02:58:25.620369 12811 heartbeater.cc:503] Master 127.8.208.126:35237 was elected leader, sending a full tablet report...
Sep 29 02:58:25 dist-test-slave-m1rv krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:58:25 dist-test-slave-m1rv krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:58:25.642987 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:25.629303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.8.208.65:34147)
0929 02:58:25.629864 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:25.629871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:25.629896 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:58:25.629955 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:25.629961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:25.630161 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:25.630295 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:25.631545 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.632526 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.633732 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.634263 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.634355 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:25.638015 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:25.638024 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:25.638038 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:25.638064 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:25.640650 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.641275 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.641284 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.641286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.641407 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.641715 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.641718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.641720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.642009 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:25.642214 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:25.642440 (+ 226us) server_negotiation.cc:299] Negotiation successful
0929 02:58:25.642651 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":282,"threads_started":1}
Sep 29 02:58:25 dist-test-slave-m1rv krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:25.659335 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:25.646903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55128 (local address 127.8.208.126:35237)
0929 02:58:25.647150 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:25.647155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:25.647243 (+ 88us) server_negotiation.cc:407] Connection header received
0929 02:58:25.647356 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:25.647360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:25.647429 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:25.647508 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:25.648551 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.649304 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.650452 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.650824 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.650918 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:25.654365 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:25.654371 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:25.654375 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:25.654415 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:25.657167 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.657763 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.657766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.657769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.657882 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.658156 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.658159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.658161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.658468 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:25.658638 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:25.658857 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 02:58:25.659002 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260929 02:58:25.659873 12622 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55128
I20260929 02:58:25.660797 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 12683
I20260929 02:58:25.672991 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:25.661703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36648 (local address 127.8.208.65:34147)
0929 02:58:25.661893 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:25.661901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:25.662187 (+ 286us) server_negotiation.cc:407] Connection header received
0929 02:58:25.662365 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:25.662379 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:25.662469 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:25.662572 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:25.663706 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.664360 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.665599 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.665996 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.666132 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:25.668142 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:25.668148 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:25.668154 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:25.668188 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:25.670595 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.671200 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.671205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.671209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.671343 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.671697 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.671703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.671706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.671938 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:25.672168 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:25.672487 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 02:58:25.672704 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:58:25.673558 12806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:25.755338 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 12683
I20260929 02:58:25.771268 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 12817
Sep 29 02:58:25 dist-test-slave-m1rv krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 02:58:25.786629 13090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:25.772224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.8.208.66:43401)
0929 02:58:25.772739 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:25.772746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:25.772771 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:58:25.772828 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:25.772835 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:25.773057 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:25.773251 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:25.774449 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.775462 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.776697 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.777364 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.777514 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:25.781571 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:25.781581 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:25.781595 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:25.781625 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:25.784384 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.784941 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.784947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.784949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.785054 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.785415 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.785418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.785420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.785716 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:25.785934 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:25.786152 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 02:58:25.786333 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":286,"threads_started":1}
I20260929 02:58:25.787846 12940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:25.870203 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 12817
I20260929 02:58:25.888671 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 12950
Sep 29 02:58:25 dist-test-slave-m1rv krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 02:58:25.904943 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:25.889712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51214 (local address 127.8.208.67:34261)
0929 02:58:25.890362 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:25.890377 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:25.890405 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:58:25.890464 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:25.890471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:25.890672 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:25.890846 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:25.892106 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.893121 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.894504 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:25.895123 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:25.895266 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:25.899440 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:25.899450 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:25.899463 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:25.899495 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:25.902229 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.902873 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.902886 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.902888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.903004 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:25.903367 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:25.903383 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:25.903390 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:25.903719 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:25.904036 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:25.904329 (+ 293us) server_negotiation.cc:299] Negotiation successful
0929 02:58:25.904604 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":343,"threads_started":1}
I20260929 02:58:25.906253 13073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:25.989120 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 12950
I20260929 02:58:26.004776 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 12590
I20260929 02:58:26.006011 12652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:26.109833 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 12590
2026-09-29T02:58:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3806 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:58:26 dist-test-slave-m1rv krb5kdc[13097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:58:26 dist-test-slave-m1rv krb5kdc[13097](info): set up 2 sockets
Sep 29 02:58:26 dist-test-slave-m1rv krb5kdc[13097](info): commencing operation
krb5kdc: starting...
W20260929 02:58:27.205658 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.001s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:58:27 dist-test-slave-m1rv krb5kdc[13097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:58:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:58:27.396409 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:42979
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:40527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:42979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:27.713297 13110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:27.713581 13110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:27.713768 13110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:27.720839 13110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:27.720973 13110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:27.721063 13110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:27.721163 13110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:27.721271 13110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:27.731069 13110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:42979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:42979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:27.732475 13110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:27.733788 13110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:27.745402 13118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:27.745428 13115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:27.745461 13116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:27.746629 13110 server_base.cc:1034] running on GCE node
I20260929 02:58:27.747438 13110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:27.748831 13110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:27.750054 13110 hybrid_clock.cc:648] HybridClock initialized: now 1790650707749996 us; error 57 us; skew 500 ppm
I20260929 02:58:27.750406 13110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:27 dist-test-slave-m1rv krb5kdc[13097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650707, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:27.753582 13110 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:58:27.754684 13110 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:27.754813 13110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:27.755127 13110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:27.758148 13110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "b71558fff079427781a222c3500abe7d"
format_stamp: "Formatted at 2026-09-29 02:58:27 on dist-test-slave-m1rv"
server_key: "851e8f85e8e9cfe296bfb8a62733282b"
server_key_iv: "1caf9a55d3f8bfe590cd6f902d02b1dd"
server_key_version: "encryptionkey@0"
I20260929 02:58:27.758931 13110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "b71558fff079427781a222c3500abe7d"
format_stamp: "Formatted at 2026-09-29 02:58:27 on dist-test-slave-m1rv"
server_key: "851e8f85e8e9cfe296bfb8a62733282b"
server_key_iv: "1caf9a55d3f8bfe590cd6f902d02b1dd"
server_key_version: "encryptionkey@0"
I20260929 02:58:27.763799 13110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:58:27.767158 13124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:27.768407 13110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:27.768584 13110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "b71558fff079427781a222c3500abe7d"
format_stamp: "Formatted at 2026-09-29 02:58:27 on dist-test-slave-m1rv"
server_key: "851e8f85e8e9cfe296bfb8a62733282b"
server_key_iv: "1caf9a55d3f8bfe590cd6f902d02b1dd"
server_key_version: "encryptionkey@0"
I20260929 02:58:27.768805 13110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:27.783522 13110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:27.787824 13110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:27.788110 13110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:27.805758 13110 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:42979
I20260929 02:58:27.805766 13175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:42979 every 8 connection(s)
I20260929 02:58:27.806619 13110 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:27.810128 13176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:27.813674 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 13110
I20260929 02:58:27.813926 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:27.814531 9025 external_mini_cluster.cc:1428] Setting key af34a5afc2c3e5c8bc95928c0d190201
I20260929 02:58:27.823144 13176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d: Bootstrap starting.
I20260929 02:58:27.826790 13176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:27.827740 13176 log.cc:824] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d: Log is configured to *not* fsync() on all Append() calls
Sep 29 02:58:27 dist-test-slave-m1rv krb5kdc[13097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:27.830818 13176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d: No bootstrap required, opened a new log
I20260929 02:58:27.838070 13179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:27.816395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36126 (local address 127.8.208.126:42979)
0929 02:58:27.817282 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:27.817297 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:27.817345 (+ 48us) server_negotiation.cc:407] Connection header received
0929 02:58:27.818972 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:27.819002 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:27.819521 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:27.819945 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:27.821201 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:27.823071 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:27.824500 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:27.825605 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:27.825796 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:27.829824 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:27.829837 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:27.829859 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:27.829919 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:27.833715 (+ 3796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:27.834359 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:27.834367 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:27.834375 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:27.834592 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:27.834917 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:27.834923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:27.834926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:27.835581 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:27.835849 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:27.836227 (+ 378us) server_negotiation.cc:299] Negotiation successful
0929 02:58:27.836685 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":358,"threads_started":1}
I20260929 02:58:27.840692 13176 raft_consensus.cc:348] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b71558fff079427781a222c3500abe7d" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42979 } }
I20260929 02:58:27.840996 13176 raft_consensus.cc:374] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:27.841066 13176 raft_consensus.cc:729] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b71558fff079427781a222c3500abe7d, State: Initialized, Role: FOLLOWER
I20260929 02:58:27.841686 13176 consensus_queue.cc:260] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b71558fff079427781a222c3500abe7d" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42979 } }
I20260929 02:58:27.841881 13176 raft_consensus.cc:388] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:27.841980 13176 raft_consensus.cc:482] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:27.842095 13176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:27.844130 13176 raft_consensus.cc:504] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b71558fff079427781a222c3500abe7d" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42979 } }
I20260929 02:58:27.844561 13176 leader_election.cc:302] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b71558fff079427781a222c3500abe7d; no voters:
I20260929 02:58:27.845081 13176 leader_election.cc:288] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:27.845244 13181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:27.845938 13181 raft_consensus.cc:686] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [term 1 LEADER]: Becoming Leader. State: Replica: b71558fff079427781a222c3500abe7d, State: Running, Role: LEADER
I20260929 02:58:27.846480 13181 consensus_queue.cc:237] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b71558fff079427781a222c3500abe7d" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42979 } }
I20260929 02:58:27.847322 13176 sys_catalog.cc:564] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:27.849339 13182 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b71558fff079427781a222c3500abe7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b71558fff079427781a222c3500abe7d" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42979 } } }
I20260929 02:58:27.849670 13182 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:27.850551 13183 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b71558fff079427781a222c3500abe7d. Latest consensus state: current_term: 1 leader_uuid: "b71558fff079427781a222c3500abe7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b71558fff079427781a222c3500abe7d" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42979 } } }
I20260929 02:58:27.850929 13183 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:27.854158 13186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:27.858530 13186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:27.866286 13186 catalog_manager.cc:1347] Generated new cluster ID: cb6edef69c8d4d70a3cb05807d0cddb9
I20260929 02:58:27.866393 13186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:27.881675 13186 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:27.883467 13186 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:27.893055 13186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d: Generated new TSK 0
I20260929 02:58:27.894239 13186 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 02:58:27.953526 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:42979
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:40527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:28.272076 13202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:28.272518 13202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:28.272702 13202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:28.280043 13202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:28.280150 13202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:28.280267 13202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:58:28.289767 13202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:42979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:28.291051 13202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:28.292387 13202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:28.304821 13207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:28.304821 13210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:28.304850 13208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:28.305482 13202 server_base.cc:1034] running on GCE node
I20260929 02:58:28.306073 13202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:28.307016 13202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:28.308240 13202 hybrid_clock.cc:648] HybridClock initialized: now 1790650708308188 us; error 40 us; skew 500 ppm
I20260929 02:58:28.308549 13202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:28 dist-test-slave-m1rv krb5kdc[13097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:28.311589 13202 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 02:58:28.312677 13202 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:28.312760 13202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:28.313004 13202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:28.315750 13202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "ac469d43486b498ea0125a99205ee73a"
format_stamp: "Formatted at 2026-09-29 02:58:28 on dist-test-slave-m1rv"
server_key: "cab24bc83e34aef45d12096a3d7a4554"
server_key_iv: "f3ff2232e96e63cca3a4dc92dcc96c32"
server_key_version: "encryptionkey@0"
I20260929 02:58:28.316506 13202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "ac469d43486b498ea0125a99205ee73a"
format_stamp: "Formatted at 2026-09-29 02:58:28 on dist-test-slave-m1rv"
server_key: "cab24bc83e34aef45d12096a3d7a4554"
server_key_iv: "f3ff2232e96e63cca3a4dc92dcc96c32"
server_key_version: "encryptionkey@0"
I20260929 02:58:28.321157 13202 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260929 02:58:28.324426 13216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:28.325757 13202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 02:58:28.325937 13202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "ac469d43486b498ea0125a99205ee73a"
format_stamp: "Formatted at 2026-09-29 02:58:28 on dist-test-slave-m1rv"
server_key: "cab24bc83e34aef45d12096a3d7a4554"
server_key_iv: "f3ff2232e96e63cca3a4dc92dcc96c32"
server_key_version: "encryptionkey@0"
I20260929 02:58:28.326143 13202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:28.341583 13202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:28.345271 13202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:28.345525 13202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:28.346401 13202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:28.347622 13202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:28.347729 13202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:28.347853 13202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:28.347961 13202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:28.380733 13202 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:46077
I20260929 02:58:28.380775 13328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:46077 every 8 connection(s)
I20260929 02:58:28.381633 13202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:58:28.382552 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 13202
I20260929 02:58:28.382792 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:58:28.383358 9025 external_mini_cluster.cc:1428] Setting key e09861e2141e84de7738234017506f7e
Sep 29 02:58:28 dist-test-slave-m1rv krb5kdc[13097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
I20260929 02:58:28.402094 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:28.385140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:43763 (local address 127.8.208.126:42979)
0929 02:58:28.385601 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:28.385609 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:28.385628 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:58:28.387471 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:28.387479 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:28.387563 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:28.387680 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:28.389110 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:28.390258 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:28.392156 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:28.392562 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:28.392747 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:28.396199 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:28.396209 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:28.396212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:28.396243 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:28.398933 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:28.399674 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:28.399679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:28.399682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:28.399803 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:28.400227 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:28.400233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:28.400236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:28.400527 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:28.400685 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:28.401489 (+ 804us) server_negotiation.cc:299] Negotiation successful
0929 02:58:28.401718 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":944,"threads_started":1}
I20260929 02:58:28.403040 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:28.384644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:42979 (local address 127.8.208.65:43763)
0929 02:58:28.385430 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:28.385482 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:28.387200 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:28.387867 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:28.387875 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:28.388383 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:28.388936 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:28.388958 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:28.390401 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:28.390407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:28.392009 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:28.392021 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:28.392822 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:28.392829 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:28.393095 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:28.393864 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:28.393886 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:28.395978 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:28.399079 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:28.399087 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:28.399103 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:28.399542 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:28.399923 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:28.399928 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:28.399932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:28.400112 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:28.400658 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:28.400665 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:28.401020 (+ 355us) client_negotiation.cc:769] Sending connection context
0929 02:58:28.401380 (+ 360us) client_negotiation.cc:240] Negotiation successful
0929 02:58:28.401929 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":579,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
I20260929 02:58:28.404640 13329 heartbeater.cc:347] Connected to a master server at 127.8.208.126:42979
I20260929 02:58:28.404973 13329 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:28.405673 13329 heartbeater.cc:511] Master 127.8.208.126:42979 requested a full tablet report, sending...
I20260929 02:58:28.407696 13141 ts_manager.cc:194] Registered new tserver with Master: ac469d43486b498ea0125a99205ee73a (127.8.208.65:46077)
I20260929 02:58:28.409123 13141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:43763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 02:58:28.426313 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:42979
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:40527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:28.737388 13335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:28.737671 13335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:28.737835 13335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:28.744973 13335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:28.745069 13335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:28.745306 13335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:58:28.755059 13335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:42979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:28.756358 13335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:28.757719 13335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:28.770586 13340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:28.770675 13343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:28.770635 13341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:28.771688 13335 server_base.cc:1034] running on GCE node
I20260929 02:58:28.772311 13335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:28.773402 13335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:28.774605 13335 hybrid_clock.cc:648] HybridClock initialized: now 1790650708774559 us; error 39 us; skew 500 ppm
I20260929 02:58:28.774935 13335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:28 dist-test-slave-m1rv krb5kdc[13097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:28.778014 13335 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 02:58:28.779110 13335 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:28.779201 13335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:28.779472 13335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:28.782409 13335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "3f453bd509da46b2b8844cfa7668e01e"
format_stamp: "Formatted at 2026-09-29 02:58:28 on dist-test-slave-m1rv"
server_key: "e26c4d4d974ea2e62aba99359df1fcc5"
server_key_iv: "b04b1324219e253bd61cdab1ccd36ab0"
server_key_version: "encryptionkey@0"
I20260929 02:58:28.783246 13335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "3f453bd509da46b2b8844cfa7668e01e"
format_stamp: "Formatted at 2026-09-29 02:58:28 on dist-test-slave-m1rv"
server_key: "e26c4d4d974ea2e62aba99359df1fcc5"
server_key_iv: "b04b1324219e253bd61cdab1ccd36ab0"
server_key_version: "encryptionkey@0"
I20260929 02:58:28.788283 13335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:58:28.791741 13349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:28.793128 13335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:28.793349 13335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "3f453bd509da46b2b8844cfa7668e01e"
format_stamp: "Formatted at 2026-09-29 02:58:28 on dist-test-slave-m1rv"
server_key: "e26c4d4d974ea2e62aba99359df1fcc5"
server_key_iv: "b04b1324219e253bd61cdab1ccd36ab0"
server_key_version: "encryptionkey@0"
I20260929 02:58:28.793572 13335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:28.816715 13335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:28.821784 13335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:28.822063 13335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:28.823163 13335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:28.824731 13335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:28.824834 13335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:28.824939 13335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:28.825006 13335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:28.858443 13335 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:41343
I20260929 02:58:28.858484 13461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:41343 every 8 connection(s)
I20260929 02:58:28.859349 13335 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:58:28.864394 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 13335
I20260929 02:58:28.864576 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:58:28.865023 9025 external_mini_cluster.cc:1428] Setting key c8466767bd6488cc0090b31fb7dbd6ef
Sep 29 02:58:28 dist-test-slave-m1rv krb5kdc[13097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:28.880640 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:28.862128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:32951 (local address 127.8.208.126:42979)
0929 02:58:28.862302 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:28.862307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:28.863418 (+ 1111us) server_negotiation.cc:407] Connection header received
0929 02:58:28.865410 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:28.865417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:28.865507 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:28.865618 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:28.867357 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:28.868084 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:28.869732 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:28.870135 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:28.870270 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:28.873899 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:28.873909 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:28.873917 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:28.873961 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:28.877457 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:28.878159 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:28.878167 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:28.878173 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:28.878329 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:28.878719 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:28.878727 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:28.878734 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:28.879113 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:28.879268 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:28.879936 (+ 668us) server_negotiation.cc:299] Negotiation successful
0929 02:58:28.880194 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:58:28.881450 13464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:28.862584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:42979 (local address 127.8.208.66:32951)
0929 02:58:28.863245 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:28.863282 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:28.865030 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:28.865855 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:28.865864 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:28.866591 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:28.867193 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:28.867209 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:28.868235 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:28.868240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:28.869585 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:28.869597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:28.870355 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:28.870362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:28.870603 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:28.871371 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:28.871393 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:28.873686 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:28.877586 (+ 3900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:28.877593 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:28.877605 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:28.878028 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:28.878436 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:28.878439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:28.878442 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:28.878602 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:28.879235 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:28.879241 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:28.879547 (+ 306us) client_negotiation.cc:769] Sending connection context
0929 02:58:28.879771 (+ 224us) client_negotiation.cc:240] Negotiation successful
0929 02:58:28.880163 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":273,"threads_started":1}
I20260929 02:58:28.883013 13462 heartbeater.cc:347] Connected to a master server at 127.8.208.126:42979
I20260929 02:58:28.883370 13462 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:28.884023 13462 heartbeater.cc:511] Master 127.8.208.126:42979 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
I20260929 02:58:28.885492 13141 ts_manager.cc:194] Registered new tserver with Master: 3f453bd509da46b2b8844cfa7668e01e (127.8.208.66:41343)
I20260929 02:58:28.886276 13141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:32951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 02:58:28.910357 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:42979
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:40527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:29.228744 13467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:29.229022 13467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:29.229197 13467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:29.236697 13467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:29.236811 13467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:29.237000 13467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:58:29.246624 13467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:42979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:29.247989 13467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:29.249346 13467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:29.266422 13472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:29.266585 13473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:29.266455 13475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:29.267371 13467 server_base.cc:1034] running on GCE node
I20260929 02:58:29.267993 13467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:29.268939 13467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:29.270166 13467 hybrid_clock.cc:648] HybridClock initialized: now 1790650709270094 us; error 89 us; skew 500 ppm
I20260929 02:58:29.270489 13467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:29 dist-test-slave-m1rv krb5kdc[13097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:29.279374 13467 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 02:58:29.280548 13467 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:29.280686 13467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:29.281023 13467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:29.283994 13467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "e9566408fed34e03aa8586de635739e5"
format_stamp: "Formatted at 2026-09-29 02:58:29 on dist-test-slave-m1rv"
server_key: "7862d6b24b6e76f00806b3b37e782f2f"
server_key_iv: "6ed462a9cfbdaf360c0217f1c0a12a49"
server_key_version: "encryptionkey@0"
I20260929 02:58:29.284817 13467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "e9566408fed34e03aa8586de635739e5"
format_stamp: "Formatted at 2026-09-29 02:58:29 on dist-test-slave-m1rv"
server_key: "7862d6b24b6e76f00806b3b37e782f2f"
server_key_iv: "6ed462a9cfbdaf360c0217f1c0a12a49"
server_key_version: "encryptionkey@0"
I20260929 02:58:29.289849 13467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 02:58:29.293144 13481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:29.294436 13467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:58:29.294612 13467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "e9566408fed34e03aa8586de635739e5"
format_stamp: "Formatted at 2026-09-29 02:58:29 on dist-test-slave-m1rv"
server_key: "7862d6b24b6e76f00806b3b37e782f2f"
server_key_iv: "6ed462a9cfbdaf360c0217f1c0a12a49"
server_key_version: "encryptionkey@0"
I20260929 02:58:29.294821 13467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:29.308036 13467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:29.311852 13467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:29.312140 13467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:29.313055 13467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:29.314363 13467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:29.314491 13467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:29.314618 13467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:29.314705 13467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:29.348214 13467 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:36303
I20260929 02:58:29.348222 13593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:36303 every 8 connection(s)
I20260929 02:58:29.349159 13467 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:58:29.358209 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 13467
I20260929 02:58:29.358395 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:58:29.358807 9025 external_mini_cluster.cc:1428] Setting key 5248fc9861445cda222c999954520505
Sep 29 02:58:29 dist-test-slave-m1rv krb5kdc[13097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:29.368263 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.351641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:44543 (local address 127.8.208.126:42979)
0929 02:58:29.351814 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:29.351818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:29.353048 (+ 1230us) server_negotiation.cc:407] Connection header received
0929 02:58:29.354935 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:29.354939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:29.355006 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:29.355098 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:29.356526 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.357321 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.359044 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.359630 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.359740 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.363408 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:29.363413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:29.363416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:29.363445 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:29.365888 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:29.366497 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:29.366500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:29.366503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:29.366606 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:29.366954 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:29.366957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:29.366959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:29.367187 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:29.367295 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:29.367894 (+ 599us) server_negotiation.cc:299] Negotiation successful
0929 02:58:29.368038 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:58:29.369338 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.352146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:42979 (local address 127.8.208.67:44543)
0929 02:58:29.352883 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:29.352931 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:29.354670 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:29.355299 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:29.355307 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:29.355807 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:29.356363 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:29.356379 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.357452 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.357455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:29.358817 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:29.358828 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.359878 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.359886 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:29.360141 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.360823 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:29.360860 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:29.363222 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:29.366004 (+ 2782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:29.366011 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:29.366022 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:29.366403 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:29.366695 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:29.366697 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:29.366700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:29.366865 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:29.367291 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:29.367296 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:29.367582 (+ 286us) client_negotiation.cc:769] Sending connection context
0929 02:58:29.367822 (+ 240us) client_negotiation.cc:240] Negotiation successful
0929 02:58:29.368178 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":314,"threads_started":1}
I20260929 02:58:29.370684 13594 heartbeater.cc:347] Connected to a master server at 127.8.208.126:42979
I20260929 02:58:29.370976 13594 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:29.371687 13594 heartbeater.cc:511] Master 127.8.208.126:42979 requested a full tablet report, sending...
I20260929 02:58:29.373237 13140 ts_manager.cc:194] Registered new tserver with Master: e9566408fed34e03aa8586de635739e5 (127.8.208.67:36303)
I20260929 02:58:29.374168 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:58:29.374181 13140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:44543
I20260929 02:58:29.390735 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.378518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.8.208.126:42979)
0929 02:58:29.378688 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:29.378692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:29.379165 (+ 473us) server_negotiation.cc:407] Connection header received
0929 02:58:29.379342 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:29.379348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:29.379412 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:29.379510 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:29.380951 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.381706 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.383165 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.383600 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.383742 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.385984 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:29.385988 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:29.385992 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:29.386019 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:29.388237 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:29.388857 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:29.388864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:29.388867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:29.388995 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:29.389407 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:29.389415 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:29.389420 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:29.389698 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:29.389954 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:29.390239 (+ 285us) server_negotiation.cc:299] Negotiation successful
0929 02:58:29.390440 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 02:58:29.394728 13140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:58:29.401746 13140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:29.411922 13329 heartbeater.cc:503] Master 127.8.208.126:42979 was elected leader, sending a full tablet report...
I20260929 02:58:29.447705 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.413587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:46077 (local address 127.0.0.1:40762)
0929 02:58:29.438066 (+ 24479us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:29.438091 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:29.438239 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:29.439895 (+ 1656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:29.439905 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:29.439937 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:29.440682 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:29.440699 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.442802 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.442809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:29.444614 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:29.444625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.446555 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.446561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:29.446757 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.446812 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 02:58:29.446985 (+ 173us) client_negotiation.cc:240] Negotiation successful
0929 02:58:29.447195 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22360,"mutex_wait_us":95,"spinlock_wait_cycles":1792,"thread_start_us":24919,"threads_started":1}
I20260929 02:58:29.448246 13611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.416274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40104 (local address 127.8.208.66:41343)
0929 02:58:29.437836 (+ 21562us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:29.437850 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:29.438945 (+ 1095us) server_negotiation.cc:407] Connection header received
0929 02:58:29.439273 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:29.439282 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:29.439559 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:29.439690 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:29.441035 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.442662 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.445669 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.447106 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.447281 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.447485 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:29.447706 (+ 221us) server_negotiation.cc:299] Negotiation successful
0929 02:58:29.447939 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21328,"thread_start_us":21169,"threads_started":1}
I20260929 02:58:29.448390 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.413604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51764 (local address 127.8.208.67:36303)
0929 02:58:29.439584 (+ 25980us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:29.439592 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:29.439627 (+ 35us) server_negotiation.cc:407] Connection header received
0929 02:58:29.439718 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:29.439724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:29.439938 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:29.440229 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:29.441332 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.442449 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.445364 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.446742 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.446881 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.447111 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:29.447888 (+ 777us) server_negotiation.cc:299] Negotiation successful
0929 02:58:29.448044 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25820,"thread_start_us":25636,"threads_started":1}
I20260929 02:58:29.448467 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.413729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40762 (local address 127.8.208.65:46077)
0929 02:58:29.439222 (+ 25493us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:29.439232 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:29.439264 (+ 32us) server_negotiation.cc:407] Connection header received
0929 02:58:29.439370 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:29.439380 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:29.439647 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:29.439853 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:29.440956 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.442674 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.444772 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.446151 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.446293 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.446523 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:29.447961 (+ 1438us) server_negotiation.cc:299] Negotiation successful
0929 02:58:29.448140 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25304,"thread_start_us":25163,"threads_started":1}
I20260929 02:58:29.448359 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.415645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:41343 (local address 127.0.0.1:40104)
0929 02:58:29.438810 (+ 23165us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:29.438831 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:29.439062 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:29.439983 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:29.439992 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:29.440025 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:29.440800 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:29.440814 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.443250 (+ 2436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.443256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:29.445493 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:29.445506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.447337 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.447346 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:29.447507 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.447540 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:58:29.447634 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 02:58:29.447762 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23019,"mutex_wait_us":61,"spinlock_wait_cycles":7680,"thread_start_us":20104,"threads_started":1}
I20260929 02:58:29.448657 13606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.413265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:36303 (local address 127.0.0.1:51764)
0929 02:58:29.427402 (+ 14137us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:29.427471 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:29.427717 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:29.440208 (+ 12491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:29.440217 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:29.440245 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:29.441046 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:29.441058 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.442594 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.442602 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:29.445144 (+ 2542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:29.445159 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.446963 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.446969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:29.447148 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.447210 (+ 62us) client_negotiation.cc:769] Sending connection context
0929 02:58:29.447309 (+ 99us) client_negotiation.cc:240] Negotiation successful
0929 02:58:29.447446 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13811,"mutex_wait_us":130,"thread_start_us":14669,"threads_started":1}
I20260929 02:58:29.452446 13397 tablet_service.cc:1467] Processing CreateTablet for tablet 3e9de515fa674ea483dc25e623b623c7 (DEFAULT_TABLE table=test-table [id=edafe21d21ed4107aa47d9935ae69781]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:29.452432 13264 tablet_service.cc:1467] Processing CreateTablet for tablet 3e9de515fa674ea483dc25e623b623c7 (DEFAULT_TABLE table=test-table [id=edafe21d21ed4107aa47d9935ae69781]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:29.452432 13529 tablet_service.cc:1467] Processing CreateTablet for tablet 3e9de515fa674ea483dc25e623b623c7 (DEFAULT_TABLE table=test-table [id=edafe21d21ed4107aa47d9935ae69781]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:29.453872 13264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e9de515fa674ea483dc25e623b623c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:29.453887 13529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e9de515fa674ea483dc25e623b623c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:29.453882 13397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e9de515fa674ea483dc25e623b623c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:29.465435 13612 tablet_bootstrap.cc:492] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e: Bootstrap starting.
I20260929 02:58:29.468854 13612 tablet_bootstrap.cc:654] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:29.469887 13614 tablet_bootstrap.cc:492] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5: Bootstrap starting.
I20260929 02:58:29.470228 13612 log.cc:824] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:29.472363 13613 tablet_bootstrap.cc:492] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a: Bootstrap starting.
I20260929 02:58:29.474706 13614 tablet_bootstrap.cc:654] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:29.475098 13613 tablet_bootstrap.cc:654] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:29.476198 13613 log.cc:824] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:29.476248 13614 log.cc:824] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:29.479511 13612 tablet_bootstrap.cc:492] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e: No bootstrap required, opened a new log
I20260929 02:58:29.479509 13614 tablet_bootstrap.cc:492] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5: No bootstrap required, opened a new log
I20260929 02:58:29.479704 13613 tablet_bootstrap.cc:492] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a: No bootstrap required, opened a new log
I20260929 02:58:29.479797 13614 ts_tablet_manager.cc:1397] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260929 02:58:29.479809 13612 ts_tablet_manager.cc:1397] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260929 02:58:29.479934 13613 ts_tablet_manager.cc:1397] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260929 02:58:29.486618 13612 raft_consensus.cc:348] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9566408fed34e03aa8586de635739e5" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36303 } } peers { permanent_uuid: "ac469d43486b498ea0125a99205ee73a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 46077 } } peers { permanent_uuid: "3f453bd509da46b2b8844cfa7668e01e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41343 } }
I20260929 02:58:29.487010 13612 raft_consensus.cc:374] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:29.487102 13612 raft_consensus.cc:729] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f453bd509da46b2b8844cfa7668e01e, State: Initialized, Role: FOLLOWER
I20260929 02:58:29.487735 13612 consensus_queue.cc:260] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9566408fed34e03aa8586de635739e5" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36303 } } peers { permanent_uuid: "ac469d43486b498ea0125a99205ee73a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 46077 } } peers { permanent_uuid: "3f453bd509da46b2b8844cfa7668e01e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41343 } }
I20260929 02:58:29.488907 13462 heartbeater.cc:503] Master 127.8.208.126:42979 was elected leader, sending a full tablet report...
I20260929 02:58:29.489639 13612 ts_tablet_manager.cc:1428] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e: Time spent starting tablet: real 0.010s user 0.006s sys 0.005s
I20260929 02:58:29.490465 13614 raft_consensus.cc:348] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9566408fed34e03aa8586de635739e5" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36303 } } peers { permanent_uuid: "ac469d43486b498ea0125a99205ee73a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 46077 } } peers { permanent_uuid: "3f453bd509da46b2b8844cfa7668e01e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41343 } }
I20260929 02:58:29.490465 13613 raft_consensus.cc:348] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9566408fed34e03aa8586de635739e5" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36303 } } peers { permanent_uuid: "ac469d43486b498ea0125a99205ee73a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 46077 } } peers { permanent_uuid: "3f453bd509da46b2b8844cfa7668e01e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41343 } }
I20260929 02:58:29.490883 13613 raft_consensus.cc:374] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:29.490896 13614 raft_consensus.cc:374] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:29.490990 13613 raft_consensus.cc:729] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac469d43486b498ea0125a99205ee73a, State: Initialized, Role: FOLLOWER
I20260929 02:58:29.490990 13614 raft_consensus.cc:729] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9566408fed34e03aa8586de635739e5, State: Initialized, Role: FOLLOWER
I20260929 02:58:29.491693 13613 consensus_queue.cc:260] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9566408fed34e03aa8586de635739e5" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36303 } } peers { permanent_uuid: "ac469d43486b498ea0125a99205ee73a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 46077 } } peers { permanent_uuid: "3f453bd509da46b2b8844cfa7668e01e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41343 } }
I20260929 02:58:29.491693 13614 consensus_queue.cc:260] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9566408fed34e03aa8586de635739e5" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36303 } } peers { permanent_uuid: "ac469d43486b498ea0125a99205ee73a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 46077 } } peers { permanent_uuid: "3f453bd509da46b2b8844cfa7668e01e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41343 } }
I20260929 02:58:29.493355 13594 heartbeater.cc:503] Master 127.8.208.126:42979 was elected leader, sending a full tablet report...
I20260929 02:58:29.494048 13614 ts_tablet_manager.cc:1428] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260929 02:58:29.495148 13613 ts_tablet_manager.cc:1428] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a: Time spent starting tablet: real 0.015s user 0.009s sys 0.007s
W20260929 02:58:29.600697 13595 tablet.cc:2369] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:58:29.611500 13463 tablet.cc:2369] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:58:29.633937 13330 tablet.cc:2369] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:58:29.763808 13618 raft_consensus.cc:482] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:29.764129 13618 raft_consensus.cc:504] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9566408fed34e03aa8586de635739e5" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36303 } } peers { permanent_uuid: "ac469d43486b498ea0125a99205ee73a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 46077 } } peers { permanent_uuid: "3f453bd509da46b2b8844cfa7668e01e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41343 } }
I20260929 02:58:29.765904 13618 leader_election.cc:288] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9566408fed34e03aa8586de635739e5 (127.8.208.67:36303), ac469d43486b498ea0125a99205ee73a (127.8.208.65:46077)
I20260929 02:58:29.774153 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.766287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:39347 (local address 127.8.208.67:36303)
0929 02:58:29.766512 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:29.766521 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:29.767529 (+ 1008us) server_negotiation.cc:407] Connection header received
0929 02:58:29.767743 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:29.767746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:29.767804 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:29.767901 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:29.768524 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.769302 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.772195 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.773314 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.773474 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.773549 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:29.773759 (+ 210us) server_negotiation.cc:299] Negotiation successful
0929 02:58:29.773895 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:58:29.774220 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.766326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:36303 (local address 127.8.208.66:39347)
0929 02:58:29.767408 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:29.767428 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:29.767598 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:29.767922 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:29.767926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:29.767957 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:29.768362 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:29.768374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.769422 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.769428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:29.772048 (+ 2620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:29.772062 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.773496 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.773500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:29.773607 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.773631 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:58:29.773715 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 02:58:29.773799 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":157,"spinlock_wait_cycles":3328,"thread_start_us":813,"threads_started":1}
I20260929 02:58:29.774840 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.766747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:34551 (local address 127.8.208.65:46077)
0929 02:58:29.767002 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:29.767011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:29.767497 (+ 486us) server_negotiation.cc:407] Connection header received
0929 02:58:29.767742 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:29.767746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:29.767802 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:29.767917 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:29.768560 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.769365 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.772666 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.773873 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.774024 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.774088 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:29.774474 (+ 386us) server_negotiation.cc:299] Negotiation successful
0929 02:58:29.774604 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:58:29.774935 13621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.766193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:46077 (local address 127.8.208.66:34551)
0929 02:58:29.767369 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:29.767399 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:29.767592 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:29.767929 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:29.767934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:29.767964 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:29.768420 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:29.768428 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.769499 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.769506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:29.772048 (+ 2542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:29.772062 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.774079 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.774086 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:29.774303 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.774333 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:58:29.774447 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 02:58:29.774566 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"mutex_wait_us":141,"spinlock_wait_cycles":55040,"thread_start_us":805,"threads_started":1}
I20260929 02:58:29.775348 13549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e9de515fa674ea483dc25e623b623c7" candidate_uuid: "3f453bd509da46b2b8844cfa7668e01e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9566408fed34e03aa8586de635739e5" is_pre_election: true
I20260929 02:58:29.775920 13549 raft_consensus.cc:2454] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f453bd509da46b2b8844cfa7668e01e in term 0.
I20260929 02:58:29.775985 13284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e9de515fa674ea483dc25e623b623c7" candidate_uuid: "3f453bd509da46b2b8844cfa7668e01e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac469d43486b498ea0125a99205ee73a" is_pre_election: true
I20260929 02:58:29.776551 13284 raft_consensus.cc:2454] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f453bd509da46b2b8844cfa7668e01e in term 0.
I20260929 02:58:29.776597 13351 leader_election.cc:302] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f453bd509da46b2b8844cfa7668e01e, e9566408fed34e03aa8586de635739e5; no voters:
I20260929 02:58:29.776989 13618 raft_consensus.cc:2781] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:29.777156 13618 raft_consensus.cc:482] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:29.777302 13618 raft_consensus.cc:3037] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:29.779405 13618 raft_consensus.cc:504] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9566408fed34e03aa8586de635739e5" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36303 } } peers { permanent_uuid: "ac469d43486b498ea0125a99205ee73a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 46077 } } peers { permanent_uuid: "3f453bd509da46b2b8844cfa7668e01e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41343 } }
I20260929 02:58:29.780009 13618 leader_election.cc:288] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [CANDIDATE]: Term 1 election: Requested vote from peers e9566408fed34e03aa8586de635739e5 (127.8.208.67:36303), ac469d43486b498ea0125a99205ee73a (127.8.208.65:46077)
I20260929 02:58:29.780452 13549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e9de515fa674ea483dc25e623b623c7" candidate_uuid: "3f453bd509da46b2b8844cfa7668e01e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9566408fed34e03aa8586de635739e5"
I20260929 02:58:29.780463 13284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e9de515fa674ea483dc25e623b623c7" candidate_uuid: "3f453bd509da46b2b8844cfa7668e01e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac469d43486b498ea0125a99205ee73a"
I20260929 02:58:29.780659 13284 raft_consensus.cc:3037] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:29.780637 13549 raft_consensus.cc:3037] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:29.782804 13549 raft_consensus.cc:2454] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f453bd509da46b2b8844cfa7668e01e in term 1.
I20260929 02:58:29.782829 13284 raft_consensus.cc:2454] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f453bd509da46b2b8844cfa7668e01e in term 1.
I20260929 02:58:29.783253 13351 leader_election.cc:302] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f453bd509da46b2b8844cfa7668e01e, e9566408fed34e03aa8586de635739e5; no voters:
I20260929 02:58:29.783581 13618 raft_consensus.cc:2781] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:29.784022 13618 raft_consensus.cc:686] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 1 LEADER]: Becoming Leader. State: Replica: 3f453bd509da46b2b8844cfa7668e01e, State: Running, Role: LEADER
I20260929 02:58:29.784498 13618 consensus_queue.cc:237] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9566408fed34e03aa8586de635739e5" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36303 } } peers { permanent_uuid: "ac469d43486b498ea0125a99205ee73a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 46077 } } peers { permanent_uuid: "3f453bd509da46b2b8844cfa7668e01e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41343 } }
I20260929 02:58:29.789213 13140 catalog_manager.cc:5495] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f453bd509da46b2b8844cfa7668e01e (127.8.208.66). New cstate: current_term: 1 leader_uuid: "3f453bd509da46b2b8844cfa7668e01e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9566408fed34e03aa8586de635739e5" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac469d43486b498ea0125a99205ee73a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 46077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f453bd509da46b2b8844cfa7668e01e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41343 } health_report { overall_health: HEALTHY } } }
I20260929 02:58:29.804912 13611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.799625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40118 (local address 127.8.208.66:41343)
0929 02:58:29.799810 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:29.799814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:29.799831 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:58:29.799974 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:29.799977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:29.800046 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:29.800163 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:29.800478 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.801149 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.802659 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.803046 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.803147 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.803641 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:29.804248 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:29.804372 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:29.804460 (+ 88us) server_negotiation.cc:299] Negotiation successful
0929 02:58:29.804528 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:58:29.813969 13284 raft_consensus.cc:1261] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a [term 1 FOLLOWER]: Refusing update from remote peer 3f453bd509da46b2b8844cfa7668e01e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:29.814124 13549 raft_consensus.cc:1261] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5 [term 1 FOLLOWER]: Refusing update from remote peer 3f453bd509da46b2b8844cfa7668e01e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:29.814970 13623 consensus_queue.cc:1035] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac469d43486b498ea0125a99205ee73a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 46077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:29.815393 13618 consensus_queue.cc:1035] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9566408fed34e03aa8586de635739e5" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:29.828753 13626 mvcc.cc:205] Tried to move back new op lower bound from 7334505307388403712 to 7334505307281178624. Current Snapshot: MvccSnapshot[applied={T|T < 7334505307388403712}]
I20260929 02:58:29.842373 13141 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:29.842629 13141 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:29.846114 13141 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b71558fff079427781a222c3500abe7d: Sending DeleteTablet for 3 replicas of tablet 3e9de515fa674ea483dc25e623b623c7
I20260929 02:58:29.847153 13264 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e9de515fa674ea483dc25e623b623c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:29 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:40762
I20260929 02:58:29.847256 13529 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e9de515fa674ea483dc25e623b623c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:29 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:51764
I20260929 02:58:29.847393 13397 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e9de515fa674ea483dc25e623b623c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:29 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:40104
I20260929 02:58:29.849759 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 13202
I20260929 02:58:29.864686 13638 tablet_replica.cc:330] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e: stopping tablet replica
I20260929 02:58:29.870317 13636 tablet_replica.cc:330] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a: stopping tablet replica
I20260929 02:58:29.873343 13638 raft_consensus.cc:2229] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:29.874142 13638 raft_consensus.cc:2258] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:29.881630 13637 tablet_replica.cc:330] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5: stopping tablet replica
I20260929 02:58:29.870743 13636 raft_consensus.cc:2229] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:29.882222 13636 raft_consensus.cc:2258] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:29.884307 13637 raft_consensus.cc:2229] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:29.884374 13638 ts_tablet_manager.cc:1905] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 02:58:29 dist-test-slave-m1rv krb5kdc[13097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:58:29.888087 13637 raft_consensus.cc:2258] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:29.890561 13637 ts_tablet_manager.cc:1905] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:29.894472 13636 ts_tablet_manager.cc:1905] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:29.895105 13638 ts_tablet_manager.cc:1918] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:29.895269 13638 log.cc:1196] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/3e9de515fa674ea483dc25e623b623c7
I20260929 02:58:29.895859 13638 ts_tablet_manager.cc:1939] T 3e9de515fa674ea483dc25e623b623c7 P 3f453bd509da46b2b8844cfa7668e01e: Deleting consensus metadata
I20260929 02:58:29.897778 13127 catalog_manager.cc:4841] TS 3f453bd509da46b2b8844cfa7668e01e (127.8.208.66:41343): tablet 3e9de515fa674ea483dc25e623b623c7 (table test-table [id=edafe21d21ed4107aa47d9935ae69781]) successfully deleted
I20260929 02:58:29.902642 13636 ts_tablet_manager.cc:1918] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:29.902839 13636 log.cc:1196] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/3e9de515fa674ea483dc25e623b623c7
I20260929 02:58:29.903366 13636 ts_tablet_manager.cc:1939] T 3e9de515fa674ea483dc25e623b623c7 P ac469d43486b498ea0125a99205ee73a: Deleting consensus metadata
I20260929 02:58:29.903730 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:29.851298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40766 (local address 127.8.208.65:46077)
0929 02:58:29.851527 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:29.851535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:29.864473 (+ 12938us) server_negotiation.cc:407] Connection header received
0929 02:58:29.864981 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:29.864990 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:29.865099 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:29.865200 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:29.866914 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.868031 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.881928 (+ 13897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:29.882580 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:29.882687 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:29.895421 (+ 12734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:29.895440 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:29.895461 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:29.895520 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:29.899681 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:29.900657 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:29.900666 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:29.900670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:29.900887 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:29.901382 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:29.901387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:29.901391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:29.901875 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:29.902210 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:29.902969 (+ 759us) server_negotiation.cc:299] Negotiation successful
0929 02:58:29.903248 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:58:29.905278 13125 catalog_manager.cc:4841] TS ac469d43486b498ea0125a99205ee73a (127.8.208.65:46077): tablet 3e9de515fa674ea483dc25e623b623c7 (table test-table [id=edafe21d21ed4107aa47d9935ae69781]) successfully deleted
I20260929 02:58:29.905427 13324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:29.909003 13637 ts_tablet_manager.cc:1918] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:29.909296 13637 log.cc:1196] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/3e9de515fa674ea483dc25e623b623c7
I20260929 02:58:29.909948 13637 ts_tablet_manager.cc:1939] T 3e9de515fa674ea483dc25e623b623c7 P e9566408fed34e03aa8586de635739e5: Deleting consensus metadata
I20260929 02:58:29.911571 13126 catalog_manager.cc:4841] TS e9566408fed34e03aa8586de635739e5 (127.8.208.67:36303): tablet 3e9de515fa674ea483dc25e623b623c7 (table test-table [id=edafe21d21ed4107aa47d9935ae69781]) successfully deleted
I20260929 02:58:29.998343 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 13202
I20260929 02:58:30.016304 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 13335
Sep 29 02:58:30 dist-test-slave-m1rv krb5kdc[13097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 02:58:30.032989 13611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:30.017230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40128 (local address 127.8.208.66:41343)
0929 02:58:30.017423 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:30.017430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:30.017452 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:58:30.017605 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:30.017611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:30.017725 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:30.017836 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:30.019014 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:30.020197 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:30.021446 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:30.022135 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:30.022268 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:30.026292 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:30.026310 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:30.026331 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:30.026370 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:30.030217 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:30.030861 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:30.030873 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:30.030878 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:30.031044 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:30.031426 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:30.031436 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:30.031439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:30.031869 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:30.032196 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:30.032370 (+ 174us) server_negotiation.cc:299] Negotiation successful
0929 02:58:30.032578 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:58:30.033756 13457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:30.146214 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 13335
I20260929 02:58:30.165022 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 13467
Sep 29 02:58:30 dist-test-slave-m1rv krb5kdc[13097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 02:58:30.180379 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:30.165877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51774 (local address 127.8.208.67:36303)
0929 02:58:30.166092 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:30.166099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:30.166120 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:58:30.166348 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:30.166353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:30.166423 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:30.166526 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:30.167817 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:30.168529 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:30.169879 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:30.170513 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:30.170646 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:30.175278 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:30.175292 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:30.175309 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:30.175341 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:30.178113 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:30.178683 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:30.178690 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:30.178692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:30.178811 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:30.179121 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:30.179124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:30.179126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:30.179465 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:30.179702 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:30.179973 (+ 271us) server_negotiation.cc:299] Negotiation successful
0929 02:58:30.180121 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260929 02:58:30.181007 13589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:30.268604 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 13467
I20260929 02:58:30.284694 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 13110
I20260929 02:58:30.285780 13171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:30.394199 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 13110
2026-09-29T02:58:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4284 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T02:58:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:30Z Disabled control of system clock
I20260929 02:58:30.458681 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:41203
--webserver_interface=127.8.208.126
--webserver_port=0
--builtin_ntp_servers=127.8.208.84:37979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:41203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 02:58:30.773269 13647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:30.773578 13647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:30.773727 13647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:30.780841 13647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:30.780941 13647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:30.780979 13647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:30.781059 13647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:30.781123 13647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:30.790252 13647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:37979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:41203
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:41203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:30.791525 13647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:30.793311 13647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:30.804894 13655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:30.804955 13652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:30.804988 13653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:30.805639 13647 server_base.cc:1034] running on GCE node
I20260929 02:58:30.806453 13647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:30.807790 13647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:30.809031 13647 hybrid_clock.cc:648] HybridClock initialized: now 1790650710808998 us; error 53 us; skew 500 ppm
I20260929 02:58:30.809432 13647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:58:30.811609 13647 webserver.cc:460] Webserver started at http://127.8.208.126:41441/ using document root <none> and password file <none>
I20260929 02:58:30.812263 13647 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:30.812431 13647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:30.812737 13647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:30.815574 13647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "a12a75b63908426dbc809e8383eb2033"
format_stamp: "Formatted at 2026-09-29 02:58:30 on dist-test-slave-m1rv"
server_key: "028a684e45b9f0899a08a3d3436122ed"
server_key_iv: "a2a25673a586c8e66a364179aab4911f"
server_key_version: "encryptionkey@0"
I20260929 02:58:30.816375 13647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "a12a75b63908426dbc809e8383eb2033"
format_stamp: "Formatted at 2026-09-29 02:58:30 on dist-test-slave-m1rv"
server_key: "028a684e45b9f0899a08a3d3436122ed"
server_key_iv: "a2a25673a586c8e66a364179aab4911f"
server_key_version: "encryptionkey@0"
I20260929 02:58:30.821257 13647 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260929 02:58:30.824441 13661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:30.825887 13647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:58:30.826092 13647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "a12a75b63908426dbc809e8383eb2033"
format_stamp: "Formatted at 2026-09-29 02:58:30 on dist-test-slave-m1rv"
server_key: "028a684e45b9f0899a08a3d3436122ed"
server_key_iv: "a2a25673a586c8e66a364179aab4911f"
server_key_version: "encryptionkey@0"
I20260929 02:58:30.826330 13647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:30.832122 13647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:30.832952 13647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:30.833398 13647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:30.852226 13647 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:41203
I20260929 02:58:30.852260 13712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:41203 every 8 connection(s)
I20260929 02:58:30.853628 13647 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:30.854756 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 13647
I20260929 02:58:30.854969 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:30.855784 9025 external_mini_cluster.cc:1428] Setting key 28a042646f93daa3b02289f9694b08c7
I20260929 02:58:30.858481 13713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:30.864317 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:30.857949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43722 (local address 127.8.208.126:41203)
0929 02:58:30.859044 (+ 1095us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:30.859056 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:30.859095 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:58:30.860674 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:30.860700 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:30.861106 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:30.861564 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:30.862659 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:30.862665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:30.862700 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:30.862714 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:30.862784 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:30.862800 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:30.863039 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:30.863186 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 02:58:30.863431 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":342,"threads_started":1}
I20260929 02:58:30.869601 13713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033: Bootstrap starting.
I20260929 02:58:30.872807 13713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:30.873914 13713 log.cc:824] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:30.876706 13713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033: No bootstrap required, opened a new log
I20260929 02:58:30.884008 13713 raft_consensus.cc:348] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12a75b63908426dbc809e8383eb2033" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 41203 } }
I20260929 02:58:30.884310 13713 raft_consensus.cc:374] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:30.884449 13713 raft_consensus.cc:729] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a12a75b63908426dbc809e8383eb2033, State: Initialized, Role: FOLLOWER
I20260929 02:58:30.884984 13713 consensus_queue.cc:260] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12a75b63908426dbc809e8383eb2033" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 41203 } }
I20260929 02:58:30.885236 13713 raft_consensus.cc:388] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:30.885382 13713 raft_consensus.cc:482] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:30.885563 13713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:30.887557 13713 raft_consensus.cc:504] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12a75b63908426dbc809e8383eb2033" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 41203 } }
I20260929 02:58:30.888093 13713 leader_election.cc:302] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a12a75b63908426dbc809e8383eb2033; no voters:
I20260929 02:58:30.888732 13713 leader_election.cc:288] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:30.888902 13718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:30.890329 13718 raft_consensus.cc:686] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [term 1 LEADER]: Becoming Leader. State: Replica: a12a75b63908426dbc809e8383eb2033, State: Running, Role: LEADER
I20260929 02:58:30.890815 13718 consensus_queue.cc:237] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12a75b63908426dbc809e8383eb2033" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 41203 } }
I20260929 02:58:30.890964 13713 sys_catalog.cc:564] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:30.893855 13720 sys_catalog.cc:455] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a12a75b63908426dbc809e8383eb2033. Latest consensus state: current_term: 1 leader_uuid: "a12a75b63908426dbc809e8383eb2033" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12a75b63908426dbc809e8383eb2033" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 41203 } } }
I20260929 02:58:30.893898 13719 sys_catalog.cc:455] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a12a75b63908426dbc809e8383eb2033" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12a75b63908426dbc809e8383eb2033" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 41203 } } }
I20260929 02:58:30.894277 13720 sys_catalog.cc:458] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:30.894274 13719 sys_catalog.cc:458] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:30.897109 13727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:30.902063 13727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:30.910346 13727 catalog_manager.cc:1347] Generated new cluster ID: 1d489159a9c9440289519cba99f697ba
I20260929 02:58:30.910475 13727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:30.926009 13727 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:30.926419 13727 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:30.939452 13727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033: Generated new TSK 0
I20260929 02:58:30.940248 13727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:58:30.949084 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:41203
--builtin_ntp_servers=127.8.208.84:37979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 02:58:31.291000 13737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:31.291332 13737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:31.291436 13737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:31.298987 13737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:31.299088 13737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:31.299197 13737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:58:31.308259 13737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:37979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:41203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:31.309557 13737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:31.311230 13737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:31.324307 13743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:31.324376 13742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:31.325039 13745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:31.325337 13737 server_base.cc:1034] running on GCE node
I20260929 02:58:31.325874 13737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:31.326807 13737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:31.328025 13737 hybrid_clock.cc:648] HybridClock initialized: now 1790650711327980 us; error 64 us; skew 500 ppm
I20260929 02:58:31.328339 13737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:58:31.330708 13737 webserver.cc:460] Webserver started at http://127.8.208.65:46829/ using document root <none> and password file <none>
I20260929 02:58:31.331414 13737 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:31.331498 13737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:31.331846 13737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:31.334760 13737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "1b8202740c844495ad724106daf96867"
format_stamp: "Formatted at 2026-09-29 02:58:31 on dist-test-slave-m1rv"
server_key: "09e72e973365ebb902f660b26338c2d5"
server_key_iv: "cf6ff4631b48de94481b7660a4301ec2"
server_key_version: "encryptionkey@0"
I20260929 02:58:31.335522 13737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "1b8202740c844495ad724106daf96867"
format_stamp: "Formatted at 2026-09-29 02:58:31 on dist-test-slave-m1rv"
server_key: "09e72e973365ebb902f660b26338c2d5"
server_key_iv: "cf6ff4631b48de94481b7660a4301ec2"
server_key_version: "encryptionkey@0"
I20260929 02:58:31.340710 13737 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:58:31.344108 13751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:31.345504 13737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 02:58:31.345659 13737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "1b8202740c844495ad724106daf96867"
format_stamp: "Formatted at 2026-09-29 02:58:31 on dist-test-slave-m1rv"
server_key: "09e72e973365ebb902f660b26338c2d5"
server_key_iv: "cf6ff4631b48de94481b7660a4301ec2"
server_key_version: "encryptionkey@0"
I20260929 02:58:31.345899 13737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:31.351224 13737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:31.351974 13737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:31.352233 13737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:31.353088 13737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:31.354406 13737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:31.354517 13737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:31.354648 13737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:31.354745 13737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:31.386229 13737 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:37209
I20260929 02:58:31.386276 13863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:37209 every 8 connection(s)
I20260929 02:58:31.387600 13737 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:58:31.395768 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:31.390152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:34455 (local address 127.8.208.126:41203)
0929 02:58:31.390575 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:31.390579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:31.391762 (+ 1183us) server_negotiation.cc:407] Connection header received
0929 02:58:31.393506 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:31.393511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:31.393579 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:31.393704 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:31.394972 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:31.394977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:31.394982 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:31.394999 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:31.395049 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:31.395062 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:31.395186 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:31.395372 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 02:58:31.395441 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":227,"threads_started":1}
I20260929 02:58:31.396736 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:31.390684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:41203 (local address 127.8.208.65:34455)
0929 02:58:31.391621 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:31.391655 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:31.393234 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:31.393935 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:31.393943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:31.394032 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:31.394685 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:58:31.394688 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:31.394725 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:58:31.394808 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:31.395156 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:31.395165 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:31.395173 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:58:31.395460 (+ 287us) client_negotiation.cc:240] Negotiation successful
0929 02:58:31.395725 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":296,"threads_started":1}
I20260929 02:58:31.397861 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 13737
I20260929 02:58:31.398061 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:58:31.398309 13864 heartbeater.cc:347] Connected to a master server at 127.8.208.126:41203
I20260929 02:58:31.398599 13864 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:31.398805 9025 external_mini_cluster.cc:1428] Setting key 23cd04bd194fc19328dc4a984912e8ff
I20260929 02:58:31.399186 13864 heartbeater.cc:511] Master 127.8.208.126:41203 requested a full tablet report, sending...
I20260929 02:58:31.401328 13678 ts_manager.cc:194] Registered new tserver with Master: 1b8202740c844495ad724106daf96867 (127.8.208.65:37209)
I20260929 02:58:31.401561 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:41203
--builtin_ntp_servers=127.8.208.84:37979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 02:58:31.724788 13868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:31.725049 13868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:31.725224 13868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:31.732666 13868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:31.732764 13868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:31.732945 13868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:58:31.742108 13868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:37979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:41203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:31.743433 13868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:31.745119 13868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:31.758298 13876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:31.758273 13874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:31.758445 13873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:31.759105 13868 server_base.cc:1034] running on GCE node
I20260929 02:58:31.759686 13868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:31.760633 13868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:31.761821 13868 hybrid_clock.cc:648] HybridClock initialized: now 1790650711761782 us; error 35 us; skew 500 ppm
I20260929 02:58:31.762131 13868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:58:31.764189 13868 webserver.cc:460] Webserver started at http://127.8.208.66:42719/ using document root <none> and password file <none>
I20260929 02:58:31.764829 13868 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:31.764950 13868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:31.765316 13868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:31.768039 13868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "114a7bdd918441be9cc07bf8530ea4ba"
format_stamp: "Formatted at 2026-09-29 02:58:31 on dist-test-slave-m1rv"
server_key: "924408b7cd60d169ae238af45839fef8"
server_key_iv: "22e0ec6afd004ddf5323290594d17b05"
server_key_version: "encryptionkey@0"
I20260929 02:58:31.768781 13868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "114a7bdd918441be9cc07bf8530ea4ba"
format_stamp: "Formatted at 2026-09-29 02:58:31 on dist-test-slave-m1rv"
server_key: "924408b7cd60d169ae238af45839fef8"
server_key_iv: "22e0ec6afd004ddf5323290594d17b05"
server_key_version: "encryptionkey@0"
I20260929 02:58:31.773779 13868 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 02:58:31.777088 13882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:31.778424 13868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:58:31.778597 13868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "114a7bdd918441be9cc07bf8530ea4ba"
format_stamp: "Formatted at 2026-09-29 02:58:31 on dist-test-slave-m1rv"
server_key: "924408b7cd60d169ae238af45839fef8"
server_key_iv: "22e0ec6afd004ddf5323290594d17b05"
server_key_version: "encryptionkey@0"
I20260929 02:58:31.778824 13868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:31.784102 13868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:31.784844 13868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:31.785117 13868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:31.785977 13868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:31.787249 13868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:31.787369 13868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:31.787505 13868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:31.787592 13868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:31.819591 13868 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:40459
I20260929 02:58:31.819620 13994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:40459 every 8 connection(s)
I20260929 02:58:31.820892 13868 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:58:31.829597 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:31.823399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:57117 (local address 127.8.208.126:41203)
0929 02:58:31.823578 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:31.823582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:31.825801 (+ 2219us) server_negotiation.cc:407] Connection header received
0929 02:58:31.827531 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:31.827536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:31.827619 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:31.827737 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:31.828916 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:31.828921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:31.828924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:31.828941 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:31.828975 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:31.828988 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:31.829091 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:31.829287 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 02:58:31.829341 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:58:31.829944 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 13868
I20260929 02:58:31.830111 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:58:31.830567 9025 external_mini_cluster.cc:1428] Setting key b86e229de74afb438409a0de7213d4d2
I20260929 02:58:31.830718 13997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:31.824014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:41203 (local address 127.8.208.66:57117)
0929 02:58:31.825657 (+ 1643us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:31.825692 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:31.827139 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:31.827927 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:31.827936 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:31.828022 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:31.828640 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:58:31.828643 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:31.828677 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:58:31.828756 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:31.829078 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:31.829085 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:31.829090 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:58:31.829348 (+ 258us) client_negotiation.cc:240] Negotiation successful
0929 02:58:31.829607 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1045,"thread_start_us":816,"threads_started":1}
I20260929 02:58:31.832129 13995 heartbeater.cc:347] Connected to a master server at 127.8.208.126:41203
I20260929 02:58:31.832437 13995 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:31.832973 13995 heartbeater.cc:511] Master 127.8.208.126:41203 requested a full tablet report, sending...
I20260929 02:58:31.833462 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:41203
--builtin_ntp_servers=127.8.208.84:37979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 02:58:31.834455 13677 ts_manager.cc:194] Registered new tserver with Master: 114a7bdd918441be9cc07bf8530ea4ba (127.8.208.66:40459)
W20260929 02:58:32.154585 13998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:32.154875 13998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:32.154965 13998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:32.162372 13998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:32.162479 13998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:32.162588 13998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:58:32.171792 13998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:37979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:41203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:32.173018 13998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:32.174667 13998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:32.187952 14003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:32.187929 14006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:32.187880 14004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:32.193568 13998 server_base.cc:1034] running on GCE node
I20260929 02:58:32.194298 13998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:32.195513 13998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:32.196733 13998 hybrid_clock.cc:648] HybridClock initialized: now 1790650712196690 us; error 34 us; skew 500 ppm
I20260929 02:58:32.197135 13998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:58:32.199604 13998 webserver.cc:460] Webserver started at http://127.8.208.67:41771/ using document root <none> and password file <none>
I20260929 02:58:32.200398 13998 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:32.200525 13998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:32.200942 13998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:32.204072 13998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "4294a287f5724e5da8937483184cc77d"
format_stamp: "Formatted at 2026-09-29 02:58:32 on dist-test-slave-m1rv"
server_key: "e69a070933de86d76a35f08129af8dc1"
server_key_iv: "a81b66d7ab27e0d1e2134a3ee2ba643a"
server_key_version: "encryptionkey@0"
I20260929 02:58:32.204919 13998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "4294a287f5724e5da8937483184cc77d"
format_stamp: "Formatted at 2026-09-29 02:58:32 on dist-test-slave-m1rv"
server_key: "e69a070933de86d76a35f08129af8dc1"
server_key_iv: "a81b66d7ab27e0d1e2134a3ee2ba643a"
server_key_version: "encryptionkey@0"
I20260929 02:58:32.210379 13998 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 02:58:32.214103 14012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:32.215457 13998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:58:32.215643 13998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "4294a287f5724e5da8937483184cc77d"
format_stamp: "Formatted at 2026-09-29 02:58:32 on dist-test-slave-m1rv"
server_key: "e69a070933de86d76a35f08129af8dc1"
server_key_iv: "a81b66d7ab27e0d1e2134a3ee2ba643a"
server_key_version: "encryptionkey@0"
I20260929 02:58:32.215867 13998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:32.221532 13998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:32.222267 13998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:32.222525 13998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:32.223405 13998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:32.224756 13998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:32.224839 13998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:32.224994 13998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:32.225083 13998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:32.257993 13998 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:45011
I20260929 02:58:32.258083 14124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:45011 every 8 connection(s)
I20260929 02:58:32.259322 13998 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:58:32.262224 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 13998
I20260929 02:58:32.262387 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:58:32.262945 9025 external_mini_cluster.cc:1428] Setting key ccb02d2319f4acfd401fdaab0385a7eb
I20260929 02:58:32.268363 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.261955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:35387 (local address 127.8.208.126:41203)
0929 02:58:32.262128 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:32.262132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:32.263935 (+ 1803us) server_negotiation.cc:407] Connection header received
0929 02:58:32.266199 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:32.266204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:32.266269 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:32.266388 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:32.267613 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:32.267619 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:32.267623 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:32.267641 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:32.267683 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:32.267694 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:32.267816 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:32.268020 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 02:58:32.268084 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:58:32.269294 14127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.262689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:41203 (local address 127.8.208.67:35387)
0929 02:58:32.263773 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.263817 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:32.265939 (+ 2122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:32.266616 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:32.266624 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:32.266712 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:32.267320 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:58:32.267322 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:32.267357 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:58:32.267452 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:32.267786 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:32.267798 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:32.267807 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 02:58:32.268085 (+ 278us) client_negotiation.cc:240] Negotiation successful
0929 02:58:32.268320 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":331,"threads_started":1}
I20260929 02:58:32.270474 14125 heartbeater.cc:347] Connected to a master server at 127.8.208.126:41203
I20260929 02:58:32.270726 14125 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:32.271226 14125 heartbeater.cc:511] Master 127.8.208.126:41203 requested a full tablet report, sending...
I20260929 02:58:32.272424 13677 ts_manager.cc:194] Registered new tserver with Master: 4294a287f5724e5da8937483184cc77d (127.8.208.67:45011)
I20260929 02:58:32.278146 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:58:32.284946 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.282603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43732 (local address 127.8.208.126:41203)
0929 02:58:32.282830 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:32.282839 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:32.283241 (+ 402us) server_negotiation.cc:407] Connection header received
0929 02:58:32.283365 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:32.283368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:32.283428 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:32.283506 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:32.284347 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:32.284351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:32.284354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:32.284368 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:32.284404 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:32.284426 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:32.284520 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:32.284600 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 02:58:32.284644 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:58:32.288606 13677 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:58:32.295455 13677 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:32.318279 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.307847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:40459 (local address 127.0.0.1:42534)
0929 02:58:32.309792 (+ 1945us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.309813 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:32.309923 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:32.313838 (+ 3915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:32.313850 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:32.313985 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:32.316365 (+ 2380us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:58:32.316371 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:32.316411 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:58:32.316546 (+ 135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:32.317267 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:32.317281 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:32.317294 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 02:58:32.317510 (+ 216us) client_negotiation.cc:240] Negotiation successful
0929 02:58:32.317634 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1769,"thread_start_us":766,"threads_started":1}
I20260929 02:58:32.320473 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.307556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42534 (local address 127.8.208.66:40459)
0929 02:58:32.311761 (+ 4205us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:32.311774 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:32.311812 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:58:32.311898 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:32.311922 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:32.312226 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:32.312411 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:32.316793 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:32.316808 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:32.316830 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:32.316866 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:32.316954 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:32.316982 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:32.317199 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:32.319918 (+ 2719us) server_negotiation.cc:299] Negotiation successful
0929 02:58:32.320024 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3978,"thread_start_us":2577,"threads_started":1}
I20260929 02:58:32.323683 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.307332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45418 (local address 127.8.208.65:37209)
0929 02:58:32.320558 (+ 13226us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:32.320566 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:32.320591 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:58:32.320651 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:32.320656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:32.321509 (+ 853us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:32.322163 (+ 654us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:32.322941 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:32.322951 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:32.322968 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:32.322982 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:32.323023 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:32.323037 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:32.323195 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:32.323268 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 02:58:32.323363 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13083,"thread_start_us":12967,"threads_started":1}
I20260929 02:58:32.323833 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.306973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:37209 (local address 127.0.0.1:45418)
0929 02:58:32.308961 (+ 1988us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.309014 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:32.309205 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:32.321709 (+ 12504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:32.321714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:32.321735 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:32.322419 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:58:32.322425 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:32.322441 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:58:32.322486 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:32.323150 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:32.323155 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:32.323160 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:58:32.323291 (+ 131us) client_negotiation.cc:240] Negotiation successful
0929 02:58:32.323345 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1624,"thread_start_us":2469,"threads_started":1}
I20260929 02:58:32.324662 14141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.307919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.8.208.67:45011)
0929 02:58:32.320339 (+ 12420us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:32.320349 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:32.320397 (+ 48us) server_negotiation.cc:407] Connection header received
0929 02:58:32.320470 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:32.320478 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:32.320720 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:32.320899 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:32.322954 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:32.322961 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:32.322973 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:32.322989 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:32.323035 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:32.323053 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:32.323216 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:32.324169 (+ 953us) server_negotiation.cc:299] Negotiation successful
0929 02:58:32.324258 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12162,"thread_start_us":12594,"threads_started":1}
I20260929 02:58:32.324630 13930 tablet_service.cc:1467] Processing CreateTablet for tablet 8c36e14c4a0d4351a26abcce62d644d2 (DEFAULT_TABLE table=test-table [id=03643e030c5a43de88ee9ab7c22878d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:32.325320 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.308240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:45011 (local address 127.0.0.1:34694)
0929 02:58:32.319463 (+ 11223us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.319487 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:32.319600 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:32.320901 (+ 1301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:32.320904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:32.320929 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:32.321676 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:58:32.321679 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:32.321696 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:58:32.321736 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:32.323276 (+ 1540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:32.323282 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:32.323286 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 02:58:32.323428 (+ 142us) client_negotiation.cc:240] Negotiation successful
0929 02:58:32.323478 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11070,"thread_start_us":11447,"threads_started":1}
I20260929 02:58:32.326097 13930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c36e14c4a0d4351a26abcce62d644d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:32.327497 13799 tablet_service.cc:1467] Processing CreateTablet for tablet 8c36e14c4a0d4351a26abcce62d644d2 (DEFAULT_TABLE table=test-table [id=03643e030c5a43de88ee9ab7c22878d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:32.328572 13799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c36e14c4a0d4351a26abcce62d644d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:32.328320 14060 tablet_service.cc:1467] Processing CreateTablet for tablet 8c36e14c4a0d4351a26abcce62d644d2 (DEFAULT_TABLE table=test-table [id=03643e030c5a43de88ee9ab7c22878d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:32.329635 14060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c36e14c4a0d4351a26abcce62d644d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:32.342417 14144 tablet_bootstrap.cc:492] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba: Bootstrap starting.
I20260929 02:58:32.344429 14145 tablet_bootstrap.cc:492] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d: Bootstrap starting.
I20260929 02:58:32.345911 14143 tablet_bootstrap.cc:492] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867: Bootstrap starting.
I20260929 02:58:32.347370 14144 tablet_bootstrap.cc:654] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:32.348354 14144 log.cc:824] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:32.348429 14145 tablet_bootstrap.cc:654] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:32.349221 14143 tablet_bootstrap.cc:654] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:32.349637 14145 log.cc:824] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:32.350451 14143 log.cc:824] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:32.351516 14144 tablet_bootstrap.cc:492] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba: No bootstrap required, opened a new log
I20260929 02:58:32.351842 14144 ts_tablet_manager.cc:1397] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260929 02:58:32.352650 14145 tablet_bootstrap.cc:492] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d: No bootstrap required, opened a new log
I20260929 02:58:32.352916 14145 ts_tablet_manager.cc:1397] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 02:58:32.354327 14143 tablet_bootstrap.cc:492] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867: No bootstrap required, opened a new log
I20260929 02:58:32.354568 14143 ts_tablet_manager.cc:1397] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 02:58:32.359794 14144 raft_consensus.cc:348] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8202740c844495ad724106daf96867" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37209 } } peers { permanent_uuid: "114a7bdd918441be9cc07bf8530ea4ba" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40459 } } peers { permanent_uuid: "4294a287f5724e5da8937483184cc77d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 45011 } }
I20260929 02:58:32.360167 14144 raft_consensus.cc:374] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:32.360270 14144 raft_consensus.cc:729] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 114a7bdd918441be9cc07bf8530ea4ba, State: Initialized, Role: FOLLOWER
I20260929 02:58:32.360890 14144 consensus_queue.cc:260] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8202740c844495ad724106daf96867" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37209 } } peers { permanent_uuid: "114a7bdd918441be9cc07bf8530ea4ba" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40459 } } peers { permanent_uuid: "4294a287f5724e5da8937483184cc77d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 45011 } }
I20260929 02:58:32.361972 13995 heartbeater.cc:503] Master 127.8.208.126:41203 was elected leader, sending a full tablet report...
I20260929 02:58:32.362610 14144 ts_tablet_manager.cc:1428] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba: Time spent starting tablet: real 0.011s user 0.005s sys 0.006s
I20260929 02:58:32.363155 14145 raft_consensus.cc:348] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8202740c844495ad724106daf96867" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37209 } } peers { permanent_uuid: "114a7bdd918441be9cc07bf8530ea4ba" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40459 } } peers { permanent_uuid: "4294a287f5724e5da8937483184cc77d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 45011 } }
I20260929 02:58:32.363595 14145 raft_consensus.cc:374] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:32.363694 14145 raft_consensus.cc:729] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4294a287f5724e5da8937483184cc77d, State: Initialized, Role: FOLLOWER
I20260929 02:58:32.364265 14145 consensus_queue.cc:260] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8202740c844495ad724106daf96867" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37209 } } peers { permanent_uuid: "114a7bdd918441be9cc07bf8530ea4ba" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40459 } } peers { permanent_uuid: "4294a287f5724e5da8937483184cc77d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 45011 } }
I20260929 02:58:32.364965 14143 raft_consensus.cc:348] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8202740c844495ad724106daf96867" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37209 } } peers { permanent_uuid: "114a7bdd918441be9cc07bf8530ea4ba" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40459 } } peers { permanent_uuid: "4294a287f5724e5da8937483184cc77d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 45011 } }
I20260929 02:58:32.365352 14125 heartbeater.cc:503] Master 127.8.208.126:41203 was elected leader, sending a full tablet report...
I20260929 02:58:32.365445 14143 raft_consensus.cc:374] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:32.365538 14143 raft_consensus.cc:729] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b8202740c844495ad724106daf96867, State: Initialized, Role: FOLLOWER
I20260929 02:58:32.365948 14145 ts_tablet_manager.cc:1428] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260929 02:58:32.366153 14143 consensus_queue.cc:260] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8202740c844495ad724106daf96867" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37209 } } peers { permanent_uuid: "114a7bdd918441be9cc07bf8530ea4ba" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40459 } } peers { permanent_uuid: "4294a287f5724e5da8937483184cc77d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 45011 } }
I20260929 02:58:32.368455 13864 heartbeater.cc:503] Master 127.8.208.126:41203 was elected leader, sending a full tablet report...
I20260929 02:58:32.369007 14143 ts_tablet_manager.cc:1428] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260929 02:58:32.372524 14151 raft_consensus.cc:482] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:32.372740 14151 raft_consensus.cc:504] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8202740c844495ad724106daf96867" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37209 } } peers { permanent_uuid: "114a7bdd918441be9cc07bf8530ea4ba" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40459 } } peers { permanent_uuid: "4294a287f5724e5da8937483184cc77d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 45011 } }
I20260929 02:58:32.374161 14151 leader_election.cc:288] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 114a7bdd918441be9cc07bf8530ea4ba (127.8.208.66:40459), 4294a287f5724e5da8937483184cc77d (127.8.208.67:45011)
I20260929 02:58:32.377657 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.374619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:37917 (local address 127.8.208.66:40459)
0929 02:58:32.374763 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:32.374767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:32.375596 (+ 829us) server_negotiation.cc:407] Connection header received
0929 02:58:32.375757 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:32.375762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:32.375820 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:32.375928 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:32.376998 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:32.377004 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:32.377008 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:32.377028 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:32.377065 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:32.377083 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:32.377245 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:32.377337 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 02:58:32.377396 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 02:58:32.377755 14141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.374432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:35535 (local address 127.8.208.67:45011)
0929 02:58:32.374614 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:32.374620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:32.375602 (+ 982us) server_negotiation.cc:407] Connection header received
0929 02:58:32.375794 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:32.375800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:32.375851 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:32.375960 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:32.377126 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:32.377155 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:32.377159 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:32.377179 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:32.377236 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:32.377251 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:32.377369 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:32.377464 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 02:58:32.377517 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:58:32.377923 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.374524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:40459 (local address 127.8.208.65:37917)
0929 02:58:32.375475 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.375497 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:32.375622 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:32.376080 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:32.376083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:32.376102 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:32.376808 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:58:32.376811 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:32.376824 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:58:32.376859 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:32.377220 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:32.377225 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:32.377230 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:58:32.377389 (+ 159us) client_negotiation.cc:240] Negotiation successful
0929 02:58:32.377458 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"thread_start_us":565,"threads_started":1}
I20260929 02:58:32.378080 14153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.374724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:45011 (local address 127.8.208.65:35535)
0929 02:58:32.375514 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.375537 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:32.375663 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:32.375982 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:32.375987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:32.376012 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:32.376929 (+ 917us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:58:32.376938 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:32.376955 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:58:32.376997 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:32.377346 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:32.377350 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:32.377354 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 02:58:32.377507 (+ 153us) client_negotiation.cc:240] Negotiation successful
0929 02:58:32.377554 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":484,"threads_started":1}
I20260929 02:58:32.378837 13950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c36e14c4a0d4351a26abcce62d644d2" candidate_uuid: "1b8202740c844495ad724106daf96867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "114a7bdd918441be9cc07bf8530ea4ba" is_pre_election: true
I20260929 02:58:32.378852 14080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c36e14c4a0d4351a26abcce62d644d2" candidate_uuid: "1b8202740c844495ad724106daf96867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4294a287f5724e5da8937483184cc77d" is_pre_election: true
I20260929 02:58:32.379290 14080 raft_consensus.cc:2454] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b8202740c844495ad724106daf96867 in term 0.
I20260929 02:58:32.379315 13950 raft_consensus.cc:2454] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b8202740c844495ad724106daf96867 in term 0.
I20260929 02:58:32.379937 13753 leader_election.cc:302] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114a7bdd918441be9cc07bf8530ea4ba, 1b8202740c844495ad724106daf96867; no voters:
I20260929 02:58:32.380321 14151 raft_consensus.cc:2781] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:32.380466 14151 raft_consensus.cc:482] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:32.380569 14151 raft_consensus.cc:3037] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:32.382771 14151 raft_consensus.cc:504] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8202740c844495ad724106daf96867" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37209 } } peers { permanent_uuid: "114a7bdd918441be9cc07bf8530ea4ba" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40459 } } peers { permanent_uuid: "4294a287f5724e5da8937483184cc77d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 45011 } }
I20260929 02:58:32.383365 14151 leader_election.cc:288] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [CANDIDATE]: Term 1 election: Requested vote from peers 114a7bdd918441be9cc07bf8530ea4ba (127.8.208.66:40459), 4294a287f5724e5da8937483184cc77d (127.8.208.67:45011)
I20260929 02:58:32.383656 13950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c36e14c4a0d4351a26abcce62d644d2" candidate_uuid: "1b8202740c844495ad724106daf96867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "114a7bdd918441be9cc07bf8530ea4ba"
I20260929 02:58:32.383827 13950 raft_consensus.cc:3037] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:32.383808 14080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c36e14c4a0d4351a26abcce62d644d2" candidate_uuid: "1b8202740c844495ad724106daf96867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4294a287f5724e5da8937483184cc77d"
I20260929 02:58:32.383987 14080 raft_consensus.cc:3037] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:32.386546 13950 raft_consensus.cc:2454] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b8202740c844495ad724106daf96867 in term 1.
I20260929 02:58:32.386685 14080 raft_consensus.cc:2454] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b8202740c844495ad724106daf96867 in term 1.
I20260929 02:58:32.386890 13753 leader_election.cc:302] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 114a7bdd918441be9cc07bf8530ea4ba, 1b8202740c844495ad724106daf96867; no voters:
I20260929 02:58:32.387166 14151 raft_consensus.cc:2781] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:32.387609 14151 raft_consensus.cc:686] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 1 LEADER]: Becoming Leader. State: Replica: 1b8202740c844495ad724106daf96867, State: Running, Role: LEADER
I20260929 02:58:32.388157 14151 consensus_queue.cc:237] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8202740c844495ad724106daf96867" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37209 } } peers { permanent_uuid: "114a7bdd918441be9cc07bf8530ea4ba" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40459 } } peers { permanent_uuid: "4294a287f5724e5da8937483184cc77d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 45011 } }
W20260929 02:58:32.389752 13865 tablet.cc:2369] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:58:32.393051 13677 catalog_manager.cc:5495] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b8202740c844495ad724106daf96867 (127.8.208.65). New cstate: current_term: 1 leader_uuid: "1b8202740c844495ad724106daf96867" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b8202740c844495ad724106daf96867" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "114a7bdd918441be9cc07bf8530ea4ba" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4294a287f5724e5da8937483184cc77d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 45011 } health_report { overall_health: UNKNOWN } } }
I20260929 02:58:32.413650 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.411399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.8.208.65:37209)
0929 02:58:32.411586 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:32.411592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:32.411706 (+ 114us) server_negotiation.cc:407] Connection header received
0929 02:58:32.411956 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:32.411960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:32.412034 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:32.412144 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:32.412959 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:32.412965 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:32.412969 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:32.412987 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:32.413023 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:32.413034 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:32.413149 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:32.413298 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 02:58:32.413354 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:58:32.421382 13950 raft_consensus.cc:1261] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba [term 1 FOLLOWER]: Refusing update from remote peer 1b8202740c844495ad724106daf96867: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:32.421360 14080 raft_consensus.cc:1261] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d [term 1 FOLLOWER]: Refusing update from remote peer 1b8202740c844495ad724106daf96867: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:32.422328 14154 consensus_queue.cc:1035] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4294a287f5724e5da8937483184cc77d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 45011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:32.422662 14151 consensus_queue.cc:1035] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "114a7bdd918441be9cc07bf8530ea4ba" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 40459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:32.447952 13677 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:32.448225 13677 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:32.451853 13677 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a12a75b63908426dbc809e8383eb2033: Sending DeleteTablet for 3 replicas of tablet 8c36e14c4a0d4351a26abcce62d644d2
I20260929 02:58:32.452992 13799 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c36e14c4a0d4351a26abcce62d644d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:32 UTC) from {username='slave'} at 127.0.0.1:45418
I20260929 02:58:32.453145 13930 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c36e14c4a0d4351a26abcce62d644d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:32 UTC) from {username='slave'} at 127.0.0.1:42534
I20260929 02:58:32.454126 14060 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c36e14c4a0d4351a26abcce62d644d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:32 UTC) from {username='slave'} at 127.0.0.1:34694
I20260929 02:58:32.454357 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 13737
I20260929 02:58:32.468910 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.456107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45444 (local address 127.8.208.65:37209)
0929 02:58:32.456280 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:32.456287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:32.466133 (+ 9846us) server_negotiation.cc:407] Connection header received
0929 02:58:32.466626 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:32.466632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:32.466697 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:32.466772 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:32.468011 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:32.468017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:32.468022 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:32.468043 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:32.468081 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:32.468101 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:32.468216 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:32.468517 (+ 301us) server_negotiation.cc:299] Negotiation successful
0929 02:58:32.468582 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:58:32.470230 13859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:32.471850 14169 tablet_replica.cc:330] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d: stopping tablet replica
I20260929 02:58:32.472467 14169 raft_consensus.cc:2229] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:32.473246 14168 tablet_replica.cc:330] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba: stopping tablet replica
I20260929 02:58:32.473781 14168 raft_consensus.cc:2229] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:32.473793 14169 raft_consensus.cc:2258] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:32.474398 14168 raft_consensus.cc:2258] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:32.476681 14169 ts_tablet_manager.cc:1905] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:32.477053 14168 ts_tablet_manager.cc:1905] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:32.485572 14168 ts_tablet_manager.cc:1918] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:32.485746 14168 log.cc:1196] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/8c36e14c4a0d4351a26abcce62d644d2
I20260929 02:58:32.486183 14168 ts_tablet_manager.cc:1939] T 8c36e14c4a0d4351a26abcce62d644d2 P 114a7bdd918441be9cc07bf8530ea4ba: Deleting consensus metadata
I20260929 02:58:32.486707 14169 ts_tablet_manager.cc:1918] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:32.486866 14169 log.cc:1196] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/8c36e14c4a0d4351a26abcce62d644d2
I20260929 02:58:32.487382 14169 ts_tablet_manager.cc:1939] T 8c36e14c4a0d4351a26abcce62d644d2 P 4294a287f5724e5da8937483184cc77d: Deleting consensus metadata
I20260929 02:58:32.489439 13662 catalog_manager.cc:4841] TS 4294a287f5724e5da8937483184cc77d (127.8.208.67:45011): tablet 8c36e14c4a0d4351a26abcce62d644d2 (table test-table [id=03643e030c5a43de88ee9ab7c22878d6]) successfully deleted
I20260929 02:58:32.489672 13663 catalog_manager.cc:4841] TS 114a7bdd918441be9cc07bf8530ea4ba (127.8.208.66:40459): tablet 8c36e14c4a0d4351a26abcce62d644d2 (table test-table [id=03643e030c5a43de88ee9ab7c22878d6]) successfully deleted
I20260929 02:58:32.574513 14167 tablet_replica.cc:330] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867: stopping tablet replica
I20260929 02:58:32.575016 14167 raft_consensus.cc:2229] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:32.575709 14167 raft_consensus.cc:2258] T 8c36e14c4a0d4351a26abcce62d644d2 P 1b8202740c844495ad724106daf96867 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:32.577062 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 13737
I20260929 02:58:32.595587 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 13868
W20260929 02:58:32.595541 13664 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.208.65:37209 (error 108)
I20260929 02:58:32.597394 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.596886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:37209 (local address 127.0.0.1:45446)
0929 02:58:32.597066 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.597229 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 02:58:32.597632 13664 catalog_manager.cc:4584] TS 1b8202740c844495ad724106daf96867 (127.8.208.65:37209): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c36e14c4a0d4351a26abcce62d644d2: Network error: Client connection negotiation failed: client connection to 127.8.208.65:37209: connect: Connection refused (error 111)
I20260929 02:58:32.598635 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.596474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.8.208.66:40459)
0929 02:58:32.596668 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:32.596674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:32.596804 (+ 130us) server_negotiation.cc:407] Connection header received
0929 02:58:32.597050 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:32.597056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:32.597151 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:32.597291 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:32.598182 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:32.598186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:32.598189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:32.598204 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:32.598234 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:32.598245 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:32.598351 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:32.598387 (+ 36us) server_negotiation.cc:299] Negotiation successful
0929 02:58:32.598441 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:58:32.599267 13990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:32.651446 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.650869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:37209 (local address 127.0.0.1:45448)
0929 02:58:32.651103 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.651229 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260929 02:58:32.652504 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.652104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:37209 (local address 127.0.0.1:45452)
0929 02:58:32.652266 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.652354 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 02:58:32.686291 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 13868
I20260929 02:58:32.702709 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 13998
I20260929 02:58:32.705878 14141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.703595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34700 (local address 127.8.208.67:45011)
0929 02:58:32.703796 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:32.703802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:32.703834 (+ 32us) server_negotiation.cc:407] Connection header received
0929 02:58:32.704013 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:32.704019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:32.704132 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:32.704246 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:32.705206 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:32.705212 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:32.705216 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:32.705234 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:32.705275 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:32.705287 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:32.705442 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:32.705520 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 02:58:32.705582 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:58:32.706579 14120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:32.713654 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.713123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:37209 (local address 127.0.0.1:45468)
0929 02:58:32.713344 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.713471 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260929 02:58:32.714696 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.714271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:37209 (local address 127.0.0.1:45476)
0929 02:58:32.714461 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.714550 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260929 02:58:32.795433 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.794915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:37209 (local address 127.0.0.1:45482)
0929 02:58:32.795123 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.795217 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 02:58:32.795385 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 13998
I20260929 02:58:32.796715 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:32.796131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:37209 (local address 127.0.0.1:45490)
0929 02:58:32.796443 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:32.796539 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260929 02:58:32.811439 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 13647
I20260929 02:58:32.812601 13708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:32.917614 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 13647
2026-09-29T02:58:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2518 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 02:58:32.954105 9025 mini_oidc.cc:152] Starting JWKS server
I20260929 02:58:32.956537 9025 webserver.cc:460] Webserver started at https://127.0.0.1:43585/ using document root <none> and password file <none>
I20260929 02:58:32.956815 9025 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 02:58:32.957448 9025 webserver.cc:460] Webserver started at http://127.0.0.1:33503/ using document root <none> and password file <none>
2026-09-29T02:58:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:32Z Disabled control of system clock
I20260929 02:58:32.982256 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37799
--webserver_interface=127.8.208.126
--webserver_port=0
--builtin_ntp_servers=127.8.208.84:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:37799
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43585/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/testchainca.cert with env {}
W20260929 02:58:33.299322 14181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:33.299607 14181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:33.299707 14181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:33.306834 14181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:33.306949 14181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:33.306986 14181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:33.307044 14181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:33.307096 14181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 02:58:33.307149 14181 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 02:58:33.307197 14181 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43585/jwks.json
I20260929 02:58:33.316901 14181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:37799
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43585/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:33.318220 14181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:33.319546 14181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:33.331650 14186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:33.331455 14189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:33.331487 14187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:33.332376 14181 server_base.cc:1034] running on GCE node
I20260929 02:58:33.333150 14181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:33.334537 14181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:33.335760 14181 hybrid_clock.cc:648] HybridClock initialized: now 1790650713335724 us; error 57 us; skew 500 ppm
I20260929 02:58:33.336139 14181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:58:33.338423 14181 webserver.cc:460] Webserver started at http://127.8.208.126:35367/ using document root <none> and password file <none>
I20260929 02:58:33.339087 14181 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:33.339226 14181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:33.339529 14181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:33.342386 14181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "799f7fce143e46088310bca303f23ee1"
format_stamp: "Formatted at 2026-09-29 02:58:33 on dist-test-slave-m1rv"
server_key: "31f0fb6012a8d9fd7a2ac8a4dfe449ab"
server_key_iv: "8d39cd2076fc0f4cc6ae64148f3feef3"
server_key_version: "encryptionkey@0"
I20260929 02:58:33.343147 14181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "799f7fce143e46088310bca303f23ee1"
format_stamp: "Formatted at 2026-09-29 02:58:33 on dist-test-slave-m1rv"
server_key: "31f0fb6012a8d9fd7a2ac8a4dfe449ab"
server_key_iv: "8d39cd2076fc0f4cc6ae64148f3feef3"
server_key_version: "encryptionkey@0"
I20260929 02:58:33.348054 14181 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260929 02:58:33.351374 14195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:33.352604 14181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 02:58:33.352774 14181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "799f7fce143e46088310bca303f23ee1"
format_stamp: "Formatted at 2026-09-29 02:58:33 on dist-test-slave-m1rv"
server_key: "31f0fb6012a8d9fd7a2ac8a4dfe449ab"
server_key_iv: "8d39cd2076fc0f4cc6ae64148f3feef3"
server_key_version: "encryptionkey@0"
I20260929 02:58:33.352984 14181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:33.385841 14181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:33.386754 14181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:33.387076 14181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:33.405366 14181 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:37799
I20260929 02:58:33.405344 14246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:37799 every 8 connection(s)
I20260929 02:58:33.406692 14181 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:33.409211 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 14181
I20260929 02:58:33.409459 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:33.410197 9025 external_mini_cluster.cc:1428] Setting key 1bdad14a3882f3d75000e28ef5ce6381
I20260929 02:58:33.411043 14247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:33.424486 14250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:33.412140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55230 (local address 127.8.208.126:37799)
0929 02:58:33.413055 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:33.413065 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:33.413107 (+ 42us) server_negotiation.cc:407] Connection header received
0929 02:58:33.415434 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:33.415455 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:33.415900 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:33.416224 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:33.416874 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:33.418299 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:33.420111 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:33.420763 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:33.420887 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:33.422217 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:33.422227 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:33.422241 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:33.422802 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:33.422875 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:33.422899 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:33.422986 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:33.423216 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 02:58:33.423569 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":321,"threads_started":1}
I20260929 02:58:33.424583 14247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1: Bootstrap starting.
I20260929 02:58:33.427868 14247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:33.428310 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:58:33.429116 14247 log.cc:824] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:33.432004 14247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1: No bootstrap required, opened a new log
I20260929 02:58:33.439019 14247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799f7fce143e46088310bca303f23ee1" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37799 } }
I20260929 02:58:33.439363 14247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:33.439458 14247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 799f7fce143e46088310bca303f23ee1, State: Initialized, Role: FOLLOWER
I20260929 02:58:33.440074 14247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799f7fce143e46088310bca303f23ee1" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37799 } }
I20260929 02:58:33.440307 14247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:33.440436 14247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:33.440623 14247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:33.442867 14247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799f7fce143e46088310bca303f23ee1" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37799 } }
I20260929 02:58:33.443369 14247 leader_election.cc:302] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 799f7fce143e46088310bca303f23ee1; no voters:
I20260929 02:58:33.443917 14247 leader_election.cc:288] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:33.444053 14252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:33.444808 14252 raft_consensus.cc:686] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [term 1 LEADER]: Becoming Leader. State: Replica: 799f7fce143e46088310bca303f23ee1, State: Running, Role: LEADER
I20260929 02:58:33.445439 14252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799f7fce143e46088310bca303f23ee1" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37799 } }
I20260929 02:58:33.446062 14247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:33.448570 14254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 799f7fce143e46088310bca303f23ee1. Latest consensus state: current_term: 1 leader_uuid: "799f7fce143e46088310bca303f23ee1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799f7fce143e46088310bca303f23ee1" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37799 } } }
I20260929 02:58:33.448580 14253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "799f7fce143e46088310bca303f23ee1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799f7fce143e46088310bca303f23ee1" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37799 } } }
I20260929 02:58:33.449245 14254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:33.449273 14253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:33.456775 14259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:33.461241 14259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:33.469976 14259 catalog_manager.cc:1347] Generated new cluster ID: daab6643b59945628329cbdf3efac5b3
I20260929 02:58:33.470095 14259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:33.479421 14259 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:33.481159 14259 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:33.497062 14259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 799f7fce143e46088310bca303f23ee1: Generated new TSK 0
I20260929 02:58:33.498144 14259 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 02:58:33.543161 14277 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 02:58:33.543335 14277 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 02:58:33.544085 14250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:33.515274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55246 (local address 127.8.208.126:37799)
0929 02:58:33.515529 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:33.515536 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:33.515973 (+ 437us) server_negotiation.cc:407] Connection header received
0929 02:58:33.516321 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:33.516325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:33.516402 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:33.516540 (+ 138us) server_negotiation.cc:226] Negotiated authn=JWT
0929 02:58:33.517238 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:33.517977 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:33.520046 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:33.520468 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:33.520562 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:33.521247 (+ 685us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 02:58:33.543549 (+ 22302us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 02:58:33.543848 (+ 299us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55246: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":63}
W20260929 02:58:33.544224 14250 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55246: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260929 02:58:33.546100 14276 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:58:33.515152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37799 (local address 127.0.0.1:55246)
0929 02:58:33.515838 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:33.515864 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:33.516077 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:33.516609 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:33.516614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:33.516627 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0929 02:58:33.517038 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:33.517052 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:33.518121 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:33.518127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:33.519892 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:33.519906 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:33.520650 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:33.520656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:33.520837 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:33.520930 (+ 93us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 02:58:33.543856 (+ 22926us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 02:58:33.544250 (+ 394us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.208.126:37799: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":338,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.208.126:37799: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 02:58:33.547614 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 14181
I20260929 02:58:33.548867 14242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:33.641367 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 14181
2026-09-29T02:58:33Z chronyd exiting
I20260929 02:58:33.690964 9025 test_util.cc:175] -----------------------------------------------
I20260929 02:58:33.691136 9025 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790650672582614-9025-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (741 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 02:58:33.695056 9025 mini_oidc.cc:152] Starting JWKS server
I20260929 02:58:33.696887 9025 webserver.cc:460] Webserver started at https://127.0.0.1:33453/ using document root <none> and password file <none>
I20260929 02:58:33.697141 9025 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 02:58:33.697798 9025 webserver.cc:460] Webserver started at http://127.0.0.1:39125/ using document root <none> and password file <none>
2026-09-29T02:58:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:33Z Disabled control of system clock
I20260929 02:58:33.722772 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:42829
--webserver_interface=127.8.208.126
--webserver_port=0
--builtin_ntp_servers=127.8.208.84:40237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:42829
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33453/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/testchainca.cert with env {}
W20260929 02:58:34.052665 14284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:34.052981 14284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:34.053143 14284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:34.060307 14284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:34.060424 14284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:34.060509 14284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:34.060601 14284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:34.060678 14284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 02:58:34.060783 14284 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 02:58:34.060838 14284 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33453/jwks.json
I20260929 02:58:34.070955 14284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:42829
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:42829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33453/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:34.072209 14284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:34.073573 14284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:34.085691 14290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:34.085902 14289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:34.086248 14292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:34.086691 14284 server_base.cc:1034] running on GCE node
I20260929 02:58:34.087390 14284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:34.088661 14284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:34.089869 14284 hybrid_clock.cc:648] HybridClock initialized: now 1790650714089832 us; error 32 us; skew 500 ppm
I20260929 02:58:34.090252 14284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:58:34.092692 14284 webserver.cc:460] Webserver started at http://127.8.208.126:46849/ using document root <none> and password file <none>
I20260929 02:58:34.093403 14284 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:34.093493 14284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:34.093767 14284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:34.096592 14284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "b6874d76f4264bee86b0c9fb6936c038"
format_stamp: "Formatted at 2026-09-29 02:58:34 on dist-test-slave-m1rv"
server_key: "0ad9b0eeedbfa6103cb7f946e3b01a94"
server_key_iv: "d1b807d2b148d84e210e8225336a1d0a"
server_key_version: "encryptionkey@0"
I20260929 02:58:34.097399 14284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "b6874d76f4264bee86b0c9fb6936c038"
format_stamp: "Formatted at 2026-09-29 02:58:34 on dist-test-slave-m1rv"
server_key: "0ad9b0eeedbfa6103cb7f946e3b01a94"
server_key_iv: "d1b807d2b148d84e210e8225336a1d0a"
server_key_version: "encryptionkey@0"
I20260929 02:58:34.102275 14284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:58:34.105623 14298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:34.106948 14284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:58:34.107116 14284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "b6874d76f4264bee86b0c9fb6936c038"
format_stamp: "Formatted at 2026-09-29 02:58:34 on dist-test-slave-m1rv"
server_key: "0ad9b0eeedbfa6103cb7f946e3b01a94"
server_key_iv: "d1b807d2b148d84e210e8225336a1d0a"
server_key_version: "encryptionkey@0"
I20260929 02:58:34.107318 14284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:34.125957 14284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:34.126888 14284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:34.127262 14284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:34.147694 14284 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:42829
I20260929 02:58:34.147701 14349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:42829 every 8 connection(s)
I20260929 02:58:34.149302 14284 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:34.153230 14350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:34.159130 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 14284
I20260929 02:58:34.159332 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:34.159952 9025 external_mini_cluster.cc:1428] Setting key 20f39ac4c7958c3a169dd36cc99a30be
I20260929 02:58:34.164537 14350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038: Bootstrap starting.
I20260929 02:58:34.168522 14350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:34.169713 14350 log.cc:824] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:34.172976 14350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038: No bootstrap required, opened a new log
I20260929 02:58:34.175508 14353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:34.161947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34510 (local address 127.8.208.126:42829)
0929 02:58:34.162993 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:34.163009 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:34.163060 (+ 51us) server_negotiation.cc:407] Connection header received
0929 02:58:34.165445 (+ 2385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:34.165473 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:34.166059 (+ 586us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:34.166573 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:34.167103 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:34.169116 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:34.170587 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:34.171453 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:34.171640 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:34.172710 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:34.172720 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:34.172738 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:34.173498 (+ 760us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:34.173596 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:34.173633 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:34.173755 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:34.173939 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 02:58:34.174242 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":299,"threads_started":1}
I20260929 02:58:34.179401 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:58:34.182281 14350 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6874d76f4264bee86b0c9fb6936c038" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42829 } }
I20260929 02:58:34.182657 14350 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:34.182785 14350 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6874d76f4264bee86b0c9fb6936c038, State: Initialized, Role: FOLLOWER
I20260929 02:58:34.183349 14350 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6874d76f4264bee86b0c9fb6936c038" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42829 } }
I20260929 02:58:34.183579 14350 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:34.183732 14350 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:34.183897 14350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:34.186110 14350 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6874d76f4264bee86b0c9fb6936c038" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42829 } }
I20260929 02:58:34.186683 14350 leader_election.cc:302] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6874d76f4264bee86b0c9fb6936c038; no voters:
I20260929 02:58:34.187325 14350 leader_election.cc:288] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:34.187425 14355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:34.188179 14355 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [term 1 LEADER]: Becoming Leader. State: Replica: b6874d76f4264bee86b0c9fb6936c038, State: Running, Role: LEADER
I20260929 02:58:34.188726 14355 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6874d76f4264bee86b0c9fb6936c038" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42829 } }
I20260929 02:58:34.189488 14350 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:34.191470 14356 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6874d76f4264bee86b0c9fb6936c038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6874d76f4264bee86b0c9fb6936c038" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42829 } } }
I20260929 02:58:34.191990 14356 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:34.192139 14357 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6874d76f4264bee86b0c9fb6936c038. Latest consensus state: current_term: 1 leader_uuid: "b6874d76f4264bee86b0c9fb6936c038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6874d76f4264bee86b0c9fb6936c038" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42829 } } }
I20260929 02:58:34.192435 14357 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:34.193763 14361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:34.200496 14361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:34.209427 14361 catalog_manager.cc:1347] Generated new cluster ID: 11410da389d043be88321672da894090
I20260929 02:58:34.209563 14361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:34.256115 14361 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:34.257616 14361 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:34.268210 14361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6874d76f4264bee86b0c9fb6936c038: Generated new TSK 0
I20260929 02:58:34.269066 14361 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 02:58:34.303220 14380 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 02:58:34.303354 14380 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260929 02:58:34.304695 14379 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:58:34.284294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:42829 (local address 127.0.0.1:34524)
0929 02:58:34.284935 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:34.284950 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:34.285066 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:34.285533 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:34.285536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:34.285545 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0929 02:58:34.285788 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:34.285796 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:34.287159 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:34.287163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:34.288274 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:34.288285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:34.289343 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:34.289349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:34.289484 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:34.289516 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 02:58:34.304200 (+ 14684us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33453/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 02:58:34.304386 (+ 186us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.208.126:42829: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33453/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":368,"threads_started":1}
I20260929 02:58:34.304682 14353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:34.284437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34524 (local address 127.8.208.126:42829)
0929 02:58:34.284649 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:34.284655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:34.285033 (+ 378us) server_negotiation.cc:407] Connection header received
0929 02:58:34.285274 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:34.285279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:34.285366 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:34.285522 (+ 156us) server_negotiation.cc:226] Negotiated authn=JWT
0929 02:58:34.285953 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:34.287046 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:34.288424 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:34.289148 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:34.289302 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:34.289807 (+ 505us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 02:58:34.303959 (+ 14152us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33453/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 02:58:34.304384 (+ 425us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34524: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33453/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":65}
W20260929 02:58:34.304847 14353 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34524: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33453/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260929 02:58:34.305893 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 14284
I20260929 02:58:34.307050 14345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:34.409313 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 14284
2026-09-29T02:58:34Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (769 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 02:58:34.464399 9025 mini_oidc.cc:152] Starting JWKS server
I20260929 02:58:34.466308 9025 webserver.cc:460] Webserver started at https://127.0.0.1:43617/ using document root <none> and password file <none>
I20260929 02:58:34.466579 9025 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 02:58:34.467198 9025 webserver.cc:460] Webserver started at http://127.0.0.1:41617/ using document root <none> and password file <none>
2026-09-29T02:58:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:34Z Disabled control of system clock
I20260929 02:58:34.493551 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:40751
--webserver_interface=127.8.208.126
--webserver_port=0
--builtin_ntp_servers=127.8.208.84:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:40751
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43617/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 02:58:34.807654 14387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:34.808002 14387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:34.808164 14387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:34.815243 14387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:34.815393 14387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:34.815493 14387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:34.815601 14387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:34.815697 14387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 02:58:34.815786 14387 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 02:58:34.815864 14387 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43617/jwks.json
I20260929 02:58:34.824642 14387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:40751
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:40751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43617/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:34.825917 14387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:34.827694 14387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:34.839365 14393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:34.839452 14395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:34.839493 14392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:34.840505 14387 server_base.cc:1034] running on GCE node
I20260929 02:58:34.841295 14387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:34.842640 14387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:34.843861 14387 hybrid_clock.cc:648] HybridClock initialized: now 1790650714843825 us; error 57 us; skew 500 ppm
I20260929 02:58:34.844199 14387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:58:34.850260 14387 webserver.cc:460] Webserver started at http://127.8.208.126:40213/ using document root <none> and password file <none>
I20260929 02:58:34.850947 14387 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:34.851080 14387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:34.851400 14387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:34.854264 14387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "342f9fcf798048ad8605b4cf75835ce7"
format_stamp: "Formatted at 2026-09-29 02:58:34 on dist-test-slave-m1rv"
server_key: "048d62e58e8a28b423b4965bf27a4903"
server_key_iv: "68e5de10064dd9425b7cfd7ec5a06b6d"
server_key_version: "encryptionkey@0"
I20260929 02:58:34.855027 14387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "342f9fcf798048ad8605b4cf75835ce7"
format_stamp: "Formatted at 2026-09-29 02:58:34 on dist-test-slave-m1rv"
server_key: "048d62e58e8a28b423b4965bf27a4903"
server_key_iv: "68e5de10064dd9425b7cfd7ec5a06b6d"
server_key_version: "encryptionkey@0"
I20260929 02:58:34.859952 14387 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:58:34.863166 14401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:34.864454 14387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:34.864631 14387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "342f9fcf798048ad8605b4cf75835ce7"
format_stamp: "Formatted at 2026-09-29 02:58:34 on dist-test-slave-m1rv"
server_key: "048d62e58e8a28b423b4965bf27a4903"
server_key_iv: "68e5de10064dd9425b7cfd7ec5a06b6d"
server_key_version: "encryptionkey@0"
I20260929 02:58:34.864835 14387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:34.881742 14387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:34.882519 14387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:34.882817 14387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:34.900455 14387 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:40751
I20260929 02:58:34.900476 14452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:40751 every 8 connection(s)
I20260929 02:58:34.901829 14387 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:34.905742 14453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:34.910362 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 14387
I20260929 02:58:34.910573 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:34.911389 9025 external_mini_cluster.cc:1428] Setting key 2ea748cfa4a0029e099ebc71d8506329
I20260929 02:58:34.917516 14453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7: Bootstrap starting.
I20260929 02:58:34.920507 14453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:34.921583 14453 log.cc:824] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:34.924253 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:34.913351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49582 (local address 127.8.208.126:40751)
0929 02:58:34.914349 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:34.914359 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:34.914401 (+ 42us) server_negotiation.cc:407] Connection header received
0929 02:58:34.916042 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:34.916063 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:34.916503 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:34.916871 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:34.917435 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:34.918744 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:34.920035 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:34.920761 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:34.920972 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:34.922113 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:34.922121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:34.922134 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:34.922677 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:34.922747 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:34.922766 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:34.922907 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:34.923107 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 02:58:34.923400 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":367,"threads_started":1}
I20260929 02:58:34.924513 14453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7: No bootstrap required, opened a new log
I20260929 02:58:34.928099 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:58:34.933269 14453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342f9fcf798048ad8605b4cf75835ce7" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40751 } }
I20260929 02:58:34.933621 14453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:34.933737 14453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342f9fcf798048ad8605b4cf75835ce7, State: Initialized, Role: FOLLOWER
I20260929 02:58:34.934403 14453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342f9fcf798048ad8605b4cf75835ce7" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40751 } }
I20260929 02:58:34.934659 14453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:34.934757 14453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:34.934877 14453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:34.936930 14453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342f9fcf798048ad8605b4cf75835ce7" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40751 } }
I20260929 02:58:34.937423 14453 leader_election.cc:302] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 342f9fcf798048ad8605b4cf75835ce7; no voters:
I20260929 02:58:34.937953 14453 leader_election.cc:288] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:34.938093 14458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:34.938812 14458 raft_consensus.cc:686] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 342f9fcf798048ad8605b4cf75835ce7, State: Running, Role: LEADER
I20260929 02:58:34.939674 14458 consensus_queue.cc:237] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342f9fcf798048ad8605b4cf75835ce7" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40751 } }
I20260929 02:58:34.940330 14453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:34.942749 14459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "342f9fcf798048ad8605b4cf75835ce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342f9fcf798048ad8605b4cf75835ce7" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40751 } } }
I20260929 02:58:34.942920 14460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 342f9fcf798048ad8605b4cf75835ce7. Latest consensus state: current_term: 1 leader_uuid: "342f9fcf798048ad8605b4cf75835ce7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342f9fcf798048ad8605b4cf75835ce7" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40751 } } }
I20260929 02:58:34.943293 14460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:34.944106 14459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:34.945973 14467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:34.950780 14467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:34.959859 14467 catalog_manager.cc:1347] Generated new cluster ID: e0a4bc99d5e2487cb5f01c78ec33611b
I20260929 02:58:34.960127 14467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:34.975214 14467 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:34.976979 14467 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:34.987661 14467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 342f9fcf798048ad8605b4cf75835ce7: Generated new TSK 0
I20260929 02:58:34.988678 14467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:58:35.010286 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:35.001655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.8.208.126:40751)
0929 02:58:35.001837 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:35.001843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:35.002216 (+ 373us) server_negotiation.cc:407] Connection header received
0929 02:58:35.002448 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:35.002454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:35.002555 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:35.002725 (+ 170us) server_negotiation.cc:226] Negotiated authn=JWT
0929 02:58:35.003167 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:35.004292 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:35.005745 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:35.006431 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:35.006565 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:35.007082 (+ 517us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 02:58:35.009615 (+ 2533us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43617/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 02:58:35.009961 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49586: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43617/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":69}
W20260929 02:58:35.010428 14456 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49586: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43617/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260929 02:58:35.012042 14482 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:58:35.001530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:40751 (local address 127.0.0.1:49586)
0929 02:58:35.002107 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:35.002123 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:35.002245 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:35.002744 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:35.002747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:35.002756 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0929 02:58:35.003021 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:35.003029 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:35.004437 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:35.004441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:35.005600 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:35.005613 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:35.006642 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:35.006647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:35.006768 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:35.006803 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 02:58:35.009849 (+ 3046us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43617/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 02:58:35.011604 (+ 1755us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.208.126:40751: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43617/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":1423,"thread_start_us":313,"threads_started":1}
I20260929 02:58:35.013607 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 14387
I20260929 02:58:35.014847 14448 generic_service.cc:196] Checking for leaks (request via RPC)
E20260929 02:58:35.025406 14483 webserver.cc:533] Webserver: sslize failed:
E20260929 02:58:35.025553 14483 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 02:58:35.107152 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 14387
2026-09-29T02:58:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (697 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T02:58:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:35Z Disabled control of system clock
I20260929 02:58:35.186200 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:35095
--webserver_interface=127.8.208.126
--webserver_port=0
--builtin_ntp_servers=127.8.208.84:35949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:35095
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/testchainca.cert with env {}
W20260929 02:58:35.502967 14488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:35.503262 14488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:35.503352 14488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:35.511242 14488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:35.511363 14488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:35.511401 14488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:35.511473 14488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:35.511518 14488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 02:58:35.511610 14488 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 02:58:35.511651 14488 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 02:58:35.521914 14488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:35949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:35095
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:35095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:35.523250 14488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:35.524500 14488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:35.535943 14496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:35.536029 14494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:35.536028 14493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:35.537372 14488 server_base.cc:1034] running on GCE node
I20260929 02:58:35.538128 14488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:35.539572 14488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:35.540803 14488 hybrid_clock.cc:648] HybridClock initialized: now 1790650715540764 us; error 48 us; skew 500 ppm
I20260929 02:58:35.541270 14488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:58:35.543672 14488 webserver.cc:460] Webserver started at http://127.8.208.126:32921/ using document root <none> and password file <none>
I20260929 02:58:35.544413 14488 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:35.544586 14488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:35.544895 14488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:35.547811 14488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "c96e1fc3c2d84de28d509a7b5b60c187"
format_stamp: "Formatted at 2026-09-29 02:58:35 on dist-test-slave-m1rv"
server_key: "d697253066708a45ba5fdb96a3b2d4d5"
server_key_iv: "10e01c3f8a40dbfe42a43cd8221ef688"
server_key_version: "encryptionkey@0"
I20260929 02:58:35.548620 14488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "c96e1fc3c2d84de28d509a7b5b60c187"
format_stamp: "Formatted at 2026-09-29 02:58:35 on dist-test-slave-m1rv"
server_key: "d697253066708a45ba5fdb96a3b2d4d5"
server_key_iv: "10e01c3f8a40dbfe42a43cd8221ef688"
server_key_version: "encryptionkey@0"
I20260929 02:58:35.553640 14488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260929 02:58:35.556805 14502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:35.558125 14488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:58:35.558313 14488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "c96e1fc3c2d84de28d509a7b5b60c187"
format_stamp: "Formatted at 2026-09-29 02:58:35 on dist-test-slave-m1rv"
server_key: "d697253066708a45ba5fdb96a3b2d4d5"
server_key_iv: "10e01c3f8a40dbfe42a43cd8221ef688"
server_key_version: "encryptionkey@0"
I20260929 02:58:35.558533 14488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:35.602190 14488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:35.603317 14488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:35.603631 14488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:35.622192 14488 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:35095
I20260929 02:58:35.622211 14553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:35095 every 8 connection(s)
I20260929 02:58:35.623556 14488 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:35.624004 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 14488
I20260929 02:58:35.624209 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:35.625072 9025 external_mini_cluster.cc:1428] Setting key fcbd0f1a4c5aa06f9075f1bc8998feff
I20260929 02:58:35.628147 14554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:35.637889 14557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:35.627037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58368 (local address 127.8.208.126:35095)
0929 02:58:35.627978 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:35.627989 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:35.628027 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:58:35.629709 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:35.629729 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:35.630161 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:35.630540 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:35.631087 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:35.632326 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:35.633608 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:35.634236 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:35.634424 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:35.635641 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:35.635648 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:35.635662 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:58:35.636203 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:35.636262 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:58:35.636280 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:35.636417 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:35.636664 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 02:58:35.636952 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":343,"threads_started":1}
I20260929 02:58:35.638772 14554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187: Bootstrap starting.
I20260929 02:58:35.641738 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:58:35.642409 14554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:35.643703 14554 log.cc:824] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:35.646473 14554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187: No bootstrap required, opened a new log
I20260929 02:58:35.653348 14554 raft_consensus.cc:348] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96e1fc3c2d84de28d509a7b5b60c187" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35095 } }
I20260929 02:58:35.653688 14554 raft_consensus.cc:374] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:35.653800 14554 raft_consensus.cc:729] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96e1fc3c2d84de28d509a7b5b60c187, State: Initialized, Role: FOLLOWER
I20260929 02:58:35.654331 14554 consensus_queue.cc:260] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96e1fc3c2d84de28d509a7b5b60c187" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35095 } }
I20260929 02:58:35.654613 14554 raft_consensus.cc:388] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:35.654769 14554 raft_consensus.cc:482] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:35.654940 14554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:35.657058 14554 raft_consensus.cc:504] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96e1fc3c2d84de28d509a7b5b60c187" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35095 } }
I20260929 02:58:35.657553 14554 leader_election.cc:302] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c96e1fc3c2d84de28d509a7b5b60c187; no voters:
I20260929 02:58:35.658116 14554 leader_election.cc:288] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:35.658241 14559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:35.659049 14559 raft_consensus.cc:686] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [term 1 LEADER]: Becoming Leader. State: Replica: c96e1fc3c2d84de28d509a7b5b60c187, State: Running, Role: LEADER
I20260929 02:58:35.659583 14559 consensus_queue.cc:237] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96e1fc3c2d84de28d509a7b5b60c187" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35095 } }
I20260929 02:58:35.660400 14554 sys_catalog.cc:564] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:35.662806 14561 sys_catalog.cc:455] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c96e1fc3c2d84de28d509a7b5b60c187. Latest consensus state: current_term: 1 leader_uuid: "c96e1fc3c2d84de28d509a7b5b60c187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96e1fc3c2d84de28d509a7b5b60c187" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35095 } } }
I20260929 02:58:35.663249 14561 sys_catalog.cc:458] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:35.664592 14565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:35.664552 14560 sys_catalog.cc:455] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c96e1fc3c2d84de28d509a7b5b60c187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96e1fc3c2d84de28d509a7b5b60c187" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35095 } } }
I20260929 02:58:35.664944 14560 sys_catalog.cc:458] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:35.668910 14565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:35.676927 14565 catalog_manager.cc:1347] Generated new cluster ID: 2be4b540257a4d5a9578bc8812199330
I20260929 02:58:35.677021 14565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:35.687353 14565 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:35.689101 14565 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:35.707877 14565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c96e1fc3c2d84de28d509a7b5b60c187: Generated new TSK 0
I20260929 02:58:35.708884 14565 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:58:35.801826 14557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:35.729735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.8.208.126:35095)
0929 02:58:35.729941 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:35.729952 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:35.730387 (+ 435us) server_negotiation.cc:407] Connection header received
0929 02:58:35.730625 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:35.730632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:35.730717 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:35.730832 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0929 02:58:35.731372 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:35.732086 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:35.734067 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:35.734506 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:35.734595 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:35.735157 (+ 562us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 02:58:35.801287 (+ 66130us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 02:58:35.801601 (+ 314us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58380: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 02:58:35.801986 14557 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58380: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260929 02:58:35.802335 14583 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:58:35.729597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35095 (local address 127.0.0.1:58380)
0929 02:58:35.730270 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:35.730290 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:35.730439 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:35.730889 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:35.730896 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:35.730909 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0929 02:58:35.731211 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:35.731235 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:35.732234 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:35.732240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:35.733935 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:35.733950 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:35.734680 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:35.734687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:35.734876 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:35.734925 (+ 49us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 02:58:35.801584 (+ 66659us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 02:58:35.801920 (+ 336us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.208.126:35095: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":374,"threads_started":1}
I20260929 02:58:35.803903 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 14488
I20260929 02:58:35.805074 14549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:35.914040 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 14488
2026-09-29T02:58:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (786 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (400 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (403 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:58:36 dist-test-slave-m1rv krb5kdc[14598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:58:36 dist-test-slave-m1rv krb5kdc[14598](info): set up 2 sockets
Sep 29 02:58:36 dist-test-slave-m1rv krb5kdc[14598](info): commencing operation
krb5kdc: starting...
W20260929 02:58:37.814160 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.004s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:58:37 dist-test-slave-m1rv krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:58:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:58:38.017418 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:44255
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:44255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:38.331076 14611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:38.331434 14611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:38.331616 14611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:38.339051 14611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:38.339197 14611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:38.339308 14611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:38.339409 14611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:38.339479 14611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:38.349220 14611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:44255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:44255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:38.350637 14611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:38.351931 14611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:38.363835 14619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:38.363900 14617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:38.363933 14616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:38.364719 14611 server_base.cc:1034] running on GCE node
I20260929 02:58:38.365517 14611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:38.366830 14611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:38.368100 14611 hybrid_clock.cc:648] HybridClock initialized: now 1790650718368063 us; error 46 us; skew 500 ppm
I20260929 02:58:38.368461 14611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:38 dist-test-slave-m1rv krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650718, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:38.371529 14611 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:58:38.372988 14611 webserver.cc:460] Webserver started at http://127.8.208.126:36737/ using document root <none> and password file <none>
I20260929 02:58:38.373698 14611 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:38.373842 14611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:38.374153 14611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:38.377092 14611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "cc373da5ad2d4598a13e3ec38c394acf"
format_stamp: "Formatted at 2026-09-29 02:58:38 on dist-test-slave-m1rv"
server_key: "15df2bb26403fc7ac4e048bf61a9756b"
server_key_iv: "a6e7478679d0585be511200e0d6daae8"
server_key_version: "encryptionkey@0"
I20260929 02:58:38.377919 14611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "cc373da5ad2d4598a13e3ec38c394acf"
format_stamp: "Formatted at 2026-09-29 02:58:38 on dist-test-slave-m1rv"
server_key: "15df2bb26403fc7ac4e048bf61a9756b"
server_key_iv: "a6e7478679d0585be511200e0d6daae8"
server_key_version: "encryptionkey@0"
I20260929 02:58:38.382671 14611 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:58:38.385897 14626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:38.387157 14611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:58:38.387334 14611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "cc373da5ad2d4598a13e3ec38c394acf"
format_stamp: "Formatted at 2026-09-29 02:58:38 on dist-test-slave-m1rv"
server_key: "15df2bb26403fc7ac4e048bf61a9756b"
server_key_iv: "a6e7478679d0585be511200e0d6daae8"
server_key_version: "encryptionkey@0"
I20260929 02:58:38.387532 14611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:38.410455 14611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:38.414968 14611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:38.415288 14611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:38.433516 14611 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:44255
I20260929 02:58:38.433566 14677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:44255 every 8 connection(s)
I20260929 02:58:38.434859 14611 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:38.436280 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 14611
I20260929 02:58:38.436573 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:38.437214 9025 external_mini_cluster.cc:1428] Setting key 3ff501984e29d650eeca62954b835f41
I20260929 02:58:38.439350 14678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:58:38 dist-test-slave-m1rv krb5kdc[14598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:38.452941 14678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf: Bootstrap starting.
I20260929 02:58:38.457259 14678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:38.458503 14678 log.cc:824] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:38.461748 14678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf: No bootstrap required, opened a new log
I20260929 02:58:38.461805 14681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:38.439199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.8.208.126:44255)
0929 02:58:38.440059 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:38.440092 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:38.440140 (+ 48us) server_negotiation.cc:407] Connection header received
0929 02:58:38.442404 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:38.442431 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:38.442958 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:38.443407 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:38.444632 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:38.446754 (+ 2122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:38.448068 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:38.449073 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:38.449285 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:38.453517 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:38.453528 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:38.453548 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:38.453589 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:38.457468 (+ 3879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:38.458084 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:38.458092 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:38.458100 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:38.458325 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:38.458691 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:38.458695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:38.458698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:38.459369 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:38.459587 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:38.460128 (+ 541us) server_negotiation.cc:299] Negotiation successful
0929 02:58:38.460525 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":362,"threads_started":1}
I20260929 02:58:38.469607 14678 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc373da5ad2d4598a13e3ec38c394acf" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 44255 } }
I20260929 02:58:38.469934 14678 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:38.470006 14678 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc373da5ad2d4598a13e3ec38c394acf, State: Initialized, Role: FOLLOWER
I20260929 02:58:38.470492 14678 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc373da5ad2d4598a13e3ec38c394acf" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 44255 } }
I20260929 02:58:38.470675 14678 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:38.470816 14678 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:38.470957 14678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:38.473358 14678 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc373da5ad2d4598a13e3ec38c394acf" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 44255 } }
I20260929 02:58:38.473816 14678 leader_election.cc:302] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc373da5ad2d4598a13e3ec38c394acf; no voters:
I20260929 02:58:38.474301 14678 leader_election.cc:288] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:38.474438 14683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:38.475095 14683 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [term 1 LEADER]: Becoming Leader. State: Replica: cc373da5ad2d4598a13e3ec38c394acf, State: Running, Role: LEADER
I20260929 02:58:38.475594 14683 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc373da5ad2d4598a13e3ec38c394acf" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 44255 } }
I20260929 02:58:38.476300 14678 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:38.479189 14685 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc373da5ad2d4598a13e3ec38c394acf. Latest consensus state: current_term: 1 leader_uuid: "cc373da5ad2d4598a13e3ec38c394acf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc373da5ad2d4598a13e3ec38c394acf" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 44255 } } }
I20260929 02:58:38.479282 14684 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc373da5ad2d4598a13e3ec38c394acf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc373da5ad2d4598a13e3ec38c394acf" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 44255 } } }
I20260929 02:58:38.479683 14685 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:38.479679 14684 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:38.481669 14691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:38.486168 14691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:38.494724 14691 catalog_manager.cc:1347] Generated new cluster ID: e290e290948d45dcac6b1221015925b0
I20260929 02:58:38.494845 14691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:38.507050 14691 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:38.508975 14691 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:38.519655 14691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc373da5ad2d4598a13e3ec38c394acf: Generated new TSK 0
I20260929 02:58:38.520750 14691 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 02:58:38.564700 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:44255
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:38.888664 14704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:38.888918 14704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:38.889006 14704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:38.896241 14704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:38.896343 14704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:38.896479 14704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:58:38.906072 14704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:44255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:38.907186 14704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:38.908406 14704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:38.921226 14710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:38.921288 14709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:38.921267 14712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:38.922068 14704 server_base.cc:1034] running on GCE node
I20260929 02:58:38.922667 14704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:38.923620 14704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:38.924855 14704 hybrid_clock.cc:648] HybridClock initialized: now 1790650718924818 us; error 57 us; skew 500 ppm
I20260929 02:58:38.925235 14704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:38 dist-test-slave-m1rv krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650718, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:38.928402 14704 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 02:58:38.929919 14704 webserver.cc:460] Webserver started at http://127.8.208.65:33415/ using document root <none> and password file <none>
I20260929 02:58:38.930586 14704 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:38.930733 14704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:38.931037 14704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:38.933934 14704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "ff528f39e86948e19107115e44595265"
format_stamp: "Formatted at 2026-09-29 02:58:38 on dist-test-slave-m1rv"
server_key: "d5455631e584af4f2cb0d8ec14f46ca8"
server_key_iv: "0d0933ae4f7108946a363c9d66c209fd"
server_key_version: "encryptionkey@0"
I20260929 02:58:38.934697 14704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "ff528f39e86948e19107115e44595265"
format_stamp: "Formatted at 2026-09-29 02:58:38 on dist-test-slave-m1rv"
server_key: "d5455631e584af4f2cb0d8ec14f46ca8"
server_key_iv: "0d0933ae4f7108946a363c9d66c209fd"
server_key_version: "encryptionkey@0"
I20260929 02:58:38.939685 14704 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:58:38.943100 14719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:38.944326 14704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:58:38.944497 14704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "ff528f39e86948e19107115e44595265"
format_stamp: "Formatted at 2026-09-29 02:58:38 on dist-test-slave-m1rv"
server_key: "d5455631e584af4f2cb0d8ec14f46ca8"
server_key_iv: "0d0933ae4f7108946a363c9d66c209fd"
server_key_version: "encryptionkey@0"
I20260929 02:58:38.944720 14704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:38.965883 14704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:38.970412 14704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:38.970721 14704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:38.971691 14704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:38.973044 14704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:38.973152 14704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:38.973335 14704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:38.973423 14704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:39.006460 14704 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:37489
I20260929 02:58:39.006520 14831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:37489 every 8 connection(s)
I20260929 02:58:39.007771 14704 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:58:39.013487 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 14704
I20260929 02:58:39.013703 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:58:39.014437 9025 external_mini_cluster.cc:1428] Setting key ff6f7c1bcfae8565069af2c63ede4682
Sep 29 02:58:39 dist-test-slave-m1rv krb5kdc[14598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650718, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:39.031066 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:39.010681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:34765 (local address 127.8.208.126:44255)
0929 02:58:39.011158 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:39.011165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:39.012188 (+ 1023us) server_negotiation.cc:407] Connection header received
0929 02:58:39.014165 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:39.014173 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:39.014274 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:39.014412 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:39.015996 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:39.016989 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:39.019780 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:39.020292 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:39.020413 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:39.024987 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:39.024997 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:39.025001 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:39.025032 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:39.027641 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:39.028585 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:39.028588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:39.028591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:39.028703 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:39.029258 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:39.029261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:39.029263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:39.029529 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:39.029637 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:39.030639 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0929 02:58:39.030803 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":243,"threads_started":1}
I20260929 02:58:39.032740 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:39.011285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:44255 (local address 127.8.208.65:34765)
0929 02:58:39.012020 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:39.012063 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:39.013864 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:39.014667 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:39.014676 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:39.015205 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:39.015807 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:39.015830 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:39.017163 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:39.017168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:39.019583 (+ 2415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:39.019603 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:39.020545 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:39.020554 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:39.020899 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:39.021745 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:39.021783 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:39.024762 (+ 2979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:39.027786 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:39.027798 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:39.027820 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:39.028463 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:39.028819 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:39.028827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:39.028833 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:39.029096 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:39.029665 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:39.029677 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:39.030199 (+ 522us) client_negotiation.cc:769] Sending connection context
0929 02:58:39.030533 (+ 334us) client_negotiation.cc:240] Negotiation successful
0929 02:58:39.031166 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":273,"threads_started":1}
I20260929 02:58:39.034358 14832 heartbeater.cc:347] Connected to a master server at 127.8.208.126:44255
I20260929 02:58:39.034703 14832 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:39.035645 14832 heartbeater.cc:511] Master 127.8.208.126:44255 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
I20260929 02:58:39.037906 14643 ts_manager.cc:194] Registered new tserver with Master: ff528f39e86948e19107115e44595265 (127.8.208.65:37489)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
I20260929 02:58:39.039685 14643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:34765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 02:58:39.064551 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:44255
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:39.388584 14838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:39.388937 14838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:39.389124 14838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:39.396545 14838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:39.396721 14838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:39.396914 14838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:58:39.406723 14838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:44255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:39.408075 14838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:39.409629 14838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:39.423560 14844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:39.423614 14843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:39.423636 14846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:39.424451 14838 server_base.cc:1034] running on GCE node
I20260929 02:58:39.425086 14838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:39.426159 14838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:39.427459 14838 hybrid_clock.cc:648] HybridClock initialized: now 1790650719427387 us; error 44 us; skew 500 ppm
I20260929 02:58:39.427891 14838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:39 dist-test-slave-m1rv krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650719, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:39.431038 14838 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 02:58:39.432539 14838 webserver.cc:460] Webserver started at http://127.8.208.66:33829/ using document root <none> and password file <none>
I20260929 02:58:39.433480 14838 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:39.433627 14838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:39.433965 14838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:39.437789 14838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "31c0527d52224fd6b6a4f8ff19bab349"
format_stamp: "Formatted at 2026-09-29 02:58:39 on dist-test-slave-m1rv"
server_key: "3dadccf6bea48e93c14c41385b38b7f7"
server_key_iv: "1e6dbf66cadc11b02b49ae3f95ce0b57"
server_key_version: "encryptionkey@0"
I20260929 02:58:39.438728 14838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "31c0527d52224fd6b6a4f8ff19bab349"
format_stamp: "Formatted at 2026-09-29 02:58:39 on dist-test-slave-m1rv"
server_key: "3dadccf6bea48e93c14c41385b38b7f7"
server_key_iv: "1e6dbf66cadc11b02b49ae3f95ce0b57"
server_key_version: "encryptionkey@0"
I20260929 02:58:39.444883 14838 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260929 02:58:39.449261 14853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:39.450835 14838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:58:39.450989 14838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "31c0527d52224fd6b6a4f8ff19bab349"
format_stamp: "Formatted at 2026-09-29 02:58:39 on dist-test-slave-m1rv"
server_key: "3dadccf6bea48e93c14c41385b38b7f7"
server_key_iv: "1e6dbf66cadc11b02b49ae3f95ce0b57"
server_key_version: "encryptionkey@0"
I20260929 02:58:39.451207 14838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:39.467619 14838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:39.471480 14838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:39.471714 14838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:39.472573 14838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:39.473824 14838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:39.473897 14838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:39.473980 14838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:39.474035 14838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:39.507048 14838 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:44405
I20260929 02:58:39.507092 14965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:44405 every 8 connection(s)
I20260929 02:58:39.508373 14838 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:58:39.513048 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 14838
I20260929 02:58:39.513257 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:58:39.513806 9025 external_mini_cluster.cc:1428] Setting key 1787e6dc948ea4b9eb666b1271129ddd
Sep 29 02:58:39 dist-test-slave-m1rv krb5kdc[14598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650719, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:39.531426 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:39.510968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:48413 (local address 127.8.208.126:44255)
0929 02:58:39.511192 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:39.511198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:39.512354 (+ 1156us) server_negotiation.cc:407] Connection header received
0929 02:58:39.514754 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:39.514759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:39.514855 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:39.514975 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:39.516750 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:39.517532 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:39.520038 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:39.520452 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:39.520551 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:39.524544 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:39.524554 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:39.524562 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:39.524608 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:39.528156 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:39.528852 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:39.528860 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:39.528867 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:39.529042 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:39.529510 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:39.529518 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:39.529523 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:39.529911 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:39.530082 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:39.530720 (+ 638us) server_negotiation.cc:299] Negotiation successful
0929 02:58:39.530988 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 02:58:39.532253 14968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:39.511478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:44255 (local address 127.8.208.66:48413)
0929 02:58:39.512181 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:39.512224 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:39.514476 (+ 2252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:39.515218 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:39.515227 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:39.515730 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:39.516563 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:39.516593 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:39.517691 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:39.517699 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:39.519889 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:39.519907 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:39.520687 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:39.520697 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:39.521040 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:39.521825 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:39.521858 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:39.524298 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:39.528300 (+ 4002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:39.528307 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:39.528320 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:39.528730 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:39.529150 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:39.529153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:39.529155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:39.529393 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:39.530039 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:39.530045 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:39.530344 (+ 299us) client_negotiation.cc:769] Sending connection context
0929 02:58:39.530572 (+ 228us) client_negotiation.cc:240] Negotiation successful
0929 02:58:39.531010 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":282,"threads_started":1}
I20260929 02:58:39.533898 14966 heartbeater.cc:347] Connected to a master server at 127.8.208.126:44255
I20260929 02:58:39.534186 14966 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:39.534893 14966 heartbeater.cc:511] Master 127.8.208.126:44255 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
I20260929 02:58:39.536365 14643 ts_manager.cc:194] Registered new tserver with Master: 31c0527d52224fd6b6a4f8ff19bab349 (127.8.208.66:44405)
I20260929 02:58:39.537263 14643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:48413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 02:58:39.564215 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:44255
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:39.886059 14971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:39.886430 14971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:39.886600 14971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:39.894529 14971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:39.894656 14971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:39.894850 14971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:58:39.904374 14971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:44255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:39.905692 14971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:39.906973 14971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:39.919998 14977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:39.919978 14976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:39.920425 14979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:39.920845 14971 server_base.cc:1034] running on GCE node
I20260929 02:58:39.921509 14971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:39.922432 14971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:39.923686 14971 hybrid_clock.cc:648] HybridClock initialized: now 1790650719923635 us; error 57 us; skew 500 ppm
I20260929 02:58:39.924000 14971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:39 dist-test-slave-m1rv krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650719, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:39.930682 14971 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 02:58:39.932111 14971 webserver.cc:460] Webserver started at http://127.8.208.67:46329/ using document root <none> and password file <none>
I20260929 02:58:39.932787 14971 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:39.932934 14971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:39.933300 14971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:39.936170 14971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "d60180ede78746e7bdeb07107a2023b1"
format_stamp: "Formatted at 2026-09-29 02:58:39 on dist-test-slave-m1rv"
server_key: "a62e0a7bcaf5db8f79757b6d154f3111"
server_key_iv: "c4aa0933c8225ff5d035df4e806397b3"
server_key_version: "encryptionkey@0"
I20260929 02:58:39.936960 14971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "d60180ede78746e7bdeb07107a2023b1"
format_stamp: "Formatted at 2026-09-29 02:58:39 on dist-test-slave-m1rv"
server_key: "a62e0a7bcaf5db8f79757b6d154f3111"
server_key_iv: "c4aa0933c8225ff5d035df4e806397b3"
server_key_version: "encryptionkey@0"
I20260929 02:58:39.941829 14971 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:58:39.945123 14986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:39.946429 14971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:58:39.946595 14971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "d60180ede78746e7bdeb07107a2023b1"
format_stamp: "Formatted at 2026-09-29 02:58:39 on dist-test-slave-m1rv"
server_key: "a62e0a7bcaf5db8f79757b6d154f3111"
server_key_iv: "c4aa0933c8225ff5d035df4e806397b3"
server_key_version: "encryptionkey@0"
I20260929 02:58:39.946789 14971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:39.967041 14971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:39.971304 14971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:39.971596 14971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:39.972517 14971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:39.973836 14971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:39.973939 14971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:39.974087 14971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:39.974159 14971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:40.007277 14971 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:41421
I20260929 02:58:40.007313 15098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:41421 every 8 connection(s)
I20260929 02:58:40.008558 14971 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:58:40.015285 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 14971
I20260929 02:58:40.015462 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:58:40.015909 9025 external_mini_cluster.cc:1428] Setting key 8c042051e0dff1a5535f51473f651b3b
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650719, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:58:40.028151 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:40.011412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:34683 (local address 127.8.208.126:44255)
0929 02:58:40.011628 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:40.011635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:40.012783 (+ 1148us) server_negotiation.cc:407] Connection header received
0929 02:58:40.014581 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:40.014585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:40.014652 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:40.014740 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:40.016375 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.017425 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.018957 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.019376 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.019513 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:40.022848 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:40.022861 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:40.022865 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:40.022897 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:40.025265 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.025958 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.025964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.025968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.026105 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.026524 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.026529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.026533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.026785 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:40.026947 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:40.027588 (+ 641us) server_negotiation.cc:299] Negotiation successful
0929 02:58:40.027796 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:58:40.029125 15101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:40.011916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:44255 (local address 127.8.208.67:34683)
0929 02:58:40.012617 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:40.012663 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:40.014329 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:40.014965 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:40.014973 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:40.015596 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:40.016199 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:40.016217 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.017584 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.017588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:40.018792 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:40.018804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.019601 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.019610 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:40.019853 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:40.020467 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:40.020493 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:40.022655 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:40.025397 (+ 2742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:40.025406 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:40.025430 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:40.025832 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:40.026217 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:40.026222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:40.026226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:40.026402 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:40.026911 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:40.026919 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:40.027238 (+ 319us) client_negotiation.cc:769] Sending connection context
0929 02:58:40.027505 (+ 267us) client_negotiation.cc:240] Negotiation successful
0929 02:58:40.027996 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":276,"threads_started":1}
I20260929 02:58:40.030567 15099 heartbeater.cc:347] Connected to a master server at 127.8.208.126:44255
I20260929 02:58:40.030894 15099 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:40.031644 15099 heartbeater.cc:511] Master 127.8.208.126:44255 requested a full tablet report, sending...
I20260929 02:58:40.033051 14643 ts_manager.cc:194] Registered new tserver with Master: d60180ede78746e7bdeb07107a2023b1 (127.8.208.67:41421)
I20260929 02:58:40.033846 14643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:34683
I20260929 02:58:40.042711 14832 heartbeater.cc:503] Master 127.8.208.126:44255 was elected leader, sending a full tablet report...
I20260929 02:58:40.043480 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/test-admin.keytab.
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:40.070430 9025 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 02:58:40.075661 15106 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:58:40.073042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.8.208.65:37489)
0929 02:58:40.073743 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:40.073750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:40.073776 (+ 26us) server_negotiation.cc:407] Connection header received
0929 02:58:40.073842 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:40.073848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:40.074052 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:40.074241 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:40.075194 (+ 953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53444: BlockingRecv error: recv got EOF from 127.0.0.1:53444 (error 108)
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":373,"threads_started":1}
W20260929 02:58:40.075758 9025 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/krb5cc' of type FILE
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:40.077454 9025 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:40.078820 9025 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:58:40.093847 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:40.080253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53458 (local address 127.8.208.65:37489)
0929 02:58:40.080460 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:40.080466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:40.080715 (+ 249us) server_negotiation.cc:407] Connection header received
0929 02:58:40.080906 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:40.080911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:40.080976 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:40.081076 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:40.082056 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.083010 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.084336 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.084982 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.085122 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:40.088409 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:40.088433 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:40.088447 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:40.088481 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:40.091146 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.091798 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.091806 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.091808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.091925 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.092295 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.092302 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.092305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.092651 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:40.092952 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:40.093316 (+ 364us) server_negotiation.cc:299] Negotiation successful
0929 02:58:40.093575 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260929 02:58:40.099462 15106 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:58:40.097764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53470 (local address 127.8.208.65:37489)
0929 02:58:40.097963 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:40.097969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:40.098177 (+ 208us) server_negotiation.cc:407] Connection header received
0929 02:58:40.098352 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:40.098357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:40.098425 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:40.098530 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:40.099222 (+ 692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53470: BlockingRecv error: recv got EOF from 127.0.0.1:53470 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260929 02:58:40.099653 9025 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/krb5cc' of type FILE
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:40.101090 9025 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:40.102403 9025 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:58:40.115305 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:40.103604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.8.208.65:37489)
0929 02:58:40.103810 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:40.103815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:40.104004 (+ 189us) server_negotiation.cc:407] Connection header received
0929 02:58:40.104109 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:40.104113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:40.104163 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:40.104239 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:40.105053 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.105793 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.106943 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.107342 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.107429 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:40.110606 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:40.110611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:40.110614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:40.110641 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:40.112839 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.113505 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.113511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.113515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.113641 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.114026 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.114032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.114036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.114294 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:40.114579 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:40.114837 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 02:58:40.115032 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260929 02:58:40.119114 15106 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:58:40.117667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53494 (local address 127.8.208.65:37489)
0929 02:58:40.117849 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:40.117854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:40.118095 (+ 241us) server_negotiation.cc:407] Connection header received
0929 02:58:40.118202 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:40.118206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:40.118264 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:40.118337 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:40.118911 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53494: BlockingRecv error: recv got EOF from 127.0.0.1:53494 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260929 02:58:40.119242 9025 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790650672582614-9025-0/krb5kdc/krb5cc' of type FILE
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:40.120626 9025 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:40.121939 9025 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:58:40.134521 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:40.123089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53496 (local address 127.8.208.65:37489)
0929 02:58:40.123272 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:40.123278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:40.123439 (+ 161us) server_negotiation.cc:407] Connection header received
0929 02:58:40.123543 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:40.123546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:40.123592 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:40.123664 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:40.124507 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.125240 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.126332 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.126713 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.126797 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:40.130078 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:40.130083 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:40.130086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:40.130111 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:40.132339 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.132867 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.132870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.132872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.133016 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.133370 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.133373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.133375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.133591 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:40.133832 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:40.134037 (+ 205us) server_negotiation.cc:299] Negotiation successful
0929 02:58:40.134208 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:58:40.136004 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 14704
I20260929 02:58:40.146622 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:40.136654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53510 (local address 127.8.208.65:37489)
0929 02:58:40.136759 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:40.136762 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:40.137167 (+ 405us) server_negotiation.cc:407] Connection header received
0929 02:58:40.137325 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:40.137328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:40.137373 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:40.137451 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:40.138224 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.138834 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.139928 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.140306 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.140391 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:40.142250 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:40.142255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:40.142258 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:40.142282 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:40.144620 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.145133 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.145136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.145138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.145318 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.145620 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.145623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.145625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.145840 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:40.146024 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:40.146260 (+ 236us) server_negotiation.cc:299] Negotiation successful
0929 02:58:40.146394 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 02:58:40.147200 14827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:40.230928 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 14704
I20260929 02:58:40.246708 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 14838
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 02:58:40.262245 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:40.247647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47650 (local address 127.8.208.66:44405)
0929 02:58:40.248160 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:40.248168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:40.248193 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:58:40.248252 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:40.248258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:40.248470 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:40.248602 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:40.249573 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.250618 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.251870 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.252400 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.252493 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:40.256505 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:40.256515 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:40.256529 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:40.256561 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:40.259407 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.260116 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.260123 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.260125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.260236 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.260604 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.260610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.260614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.260941 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:40.261272 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:40.261690 (+ 418us) server_negotiation.cc:299] Negotiation successful
0929 02:58:40.261937 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":263,"threads_started":1}
I20260929 02:58:40.263572 14961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:40.345436 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 14838
I20260929 02:58:40.360962 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 14971
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[14598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 02:58:40.376053 15121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:40.361920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45122 (local address 127.8.208.67:41421)
0929 02:58:40.362506 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:40.362513 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:40.362540 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:58:40.362598 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:40.362604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:40.362806 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:40.362987 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:40.364028 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.365042 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.366312 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:40.366843 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:40.366991 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:40.370967 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:40.370977 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:40.370990 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:40.371020 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:40.373708 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.374334 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.374340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.374342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.374455 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:40.374751 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:40.374754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:40.374756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:40.375072 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:40.375346 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:40.375508 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 02:58:40.375742 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":277,"threads_started":1}
I20260929 02:58:40.377497 15094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:40.460606 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 14971
I20260929 02:58:40.475798 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 14611
I20260929 02:58:40.477108 14673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:40.580245 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 14611
2026-09-29T02:58:40Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3867 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[15126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[15126](info): set up 2 sockets
Sep 29 02:58:40 dist-test-slave-m1rv krb5kdc[15126](info): commencing operation
krb5kdc: starting...
W20260929 02:58:41.681924 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:58:41 dist-test-slave-m1rv krb5kdc[15126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:58:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:41Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.126.keytab.
Entry for principal oryx/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.126.keytab.
I20260929 02:58:41.877848 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:33867
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.126.keytab
--principal=oryx/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:33867
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:42.191332 15139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:42.191612 15139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:42.191774 15139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:42.198907 15139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:42.199039 15139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:42.199126 15139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:42.199224 15139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:42.199306 15139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:42.209148 15139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:33867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.126.keytab
--principal=oryx/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:33867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:42.210469 15139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:42.211753 15139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:42.223939 15144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:42.224052 15145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:42.223939 15147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:42.224857 15139 server_base.cc:1034] running on GCE node
I20260929 02:58:42.225683 15139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:42.226967 15139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:42.228158 15139 hybrid_clock.cc:648] HybridClock initialized: now 1790650722228099 us; error 51 us; skew 500 ppm
I20260929 02:58:42.228500 15139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:42 dist-test-slave-m1rv krb5kdc[15126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650722, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:42.231611 15139 init.cc:377] Logged in from keytab as oryx/127.8.208.126@KRBTEST.COM (short username oryx)
I20260929 02:58:42.233021 15139 webserver.cc:460] Webserver started at http://127.8.208.126:36005/ using document root <none> and password file <none>
I20260929 02:58:42.233743 15139 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:42.233893 15139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:42.234206 15139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:42.237113 15139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "802fa531790e45aab9bdccc54d19ae50"
format_stamp: "Formatted at 2026-09-29 02:58:42 on dist-test-slave-m1rv"
server_key: "373d14c5650d1e2536aa8b828ed5c527"
server_key_iv: "3a07f57f7a6eb37e663e7f806c63831a"
server_key_version: "encryptionkey@0"
I20260929 02:58:42.237936 15139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "802fa531790e45aab9bdccc54d19ae50"
format_stamp: "Formatted at 2026-09-29 02:58:42 on dist-test-slave-m1rv"
server_key: "373d14c5650d1e2536aa8b828ed5c527"
server_key_iv: "3a07f57f7a6eb37e663e7f806c63831a"
server_key_version: "encryptionkey@0"
I20260929 02:58:42.242937 15139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:58:42.246366 15154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:42.247694 15139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260929 02:58:42.247903 15139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "802fa531790e45aab9bdccc54d19ae50"
format_stamp: "Formatted at 2026-09-29 02:58:42 on dist-test-slave-m1rv"
server_key: "373d14c5650d1e2536aa8b828ed5c527"
server_key_iv: "3a07f57f7a6eb37e663e7f806c63831a"
server_key_version: "encryptionkey@0"
I20260929 02:58:42.248133 15139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:42.281272 15139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:42.285243 15139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:42.285550 15139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:42.303687 15139 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:33867
I20260929 02:58:42.303709 15205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:33867 every 8 connection(s)
I20260929 02:58:42.305106 15139 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:42.306561 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 15139
I20260929 02:58:42.306770 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:42.307430 9025 external_mini_cluster.cc:1428] Setting key 1d173eef4f27340f1c80a1a8a4ffef0d
I20260929 02:58:42.309337 15206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:58:42 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
I20260929 02:58:42.322907 15206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50: Bootstrap starting.
I20260929 02:58:42.327126 15206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:42.328408 15206 log.cc:824] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:42.331420 15206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50: No bootstrap required, opened a new log
I20260929 02:58:42.332002 15209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:42.309553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35592 (local address 127.8.208.126:33867)
0929 02:58:42.310343 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:42.310358 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:42.310417 (+ 59us) server_negotiation.cc:407] Connection header received
0929 02:58:42.312352 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:42.312378 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:42.312885 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:42.313333 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:42.314605 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:42.316647 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:42.317987 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:42.319027 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:42.319217 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:42.323478 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:42.323488 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:42.323509 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:42.323550 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:42.327804 (+ 4254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:42.328485 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:42.328494 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:42.328504 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:42.328693 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:42.329126 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:42.329131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:42.329134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:42.329752 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:42.330100 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:42.330553 (+ 453us) server_negotiation.cc:299] Negotiation successful
0929 02:58:42.330958 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":320,"threads_started":1}
I20260929 02:58:42.339228 15206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } }
I20260929 02:58:42.339576 15206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:42.339691 15206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802fa531790e45aab9bdccc54d19ae50, State: Initialized, Role: FOLLOWER
I20260929 02:58:42.340276 15206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } }
I20260929 02:58:42.340528 15206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:42.340651 15206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:42.340823 15206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:42.343144 15206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } }
I20260929 02:58:42.343634 15206 leader_election.cc:302] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 802fa531790e45aab9bdccc54d19ae50; no voters:
I20260929 02:58:42.344189 15206 leader_election.cc:288] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:42.344319 15211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:42.345000 15211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 1 LEADER]: Becoming Leader. State: Replica: 802fa531790e45aab9bdccc54d19ae50, State: Running, Role: LEADER
I20260929 02:58:42.345544 15211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } }
I20260929 02:58:42.346239 15206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:42.348425 15212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } }
I20260929 02:58:42.348510 15213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 802fa531790e45aab9bdccc54d19ae50. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } }
I20260929 02:58:42.348829 15212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:42.348912 15213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:42.353863 15220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:42.357999 15220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:42.366648 15220 catalog_manager.cc:1347] Generated new cluster ID: b7690fbfec114c13b738f2f439821c9c
I20260929 02:58:42.366742 15220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:42.392001 15220 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:42.393525 15220 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:42.402233 15220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50: Generated new TSK 0
I20260929 02:58:42.403076 15220 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab.
Entry for principal oryx/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab.
I20260929 02:58:42.453163 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:33867
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab
--principal=oryx/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:42.774540 15232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:42.774787 15232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:42.774895 15232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:42.782002 15232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:42.782125 15232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:42.782325 15232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:58:42.791740 15232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab
--principal=oryx/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:33867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:42.792958 15232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:42.794281 15232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:42.807394 15237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:42.807504 15240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:42.807366 15238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:42.808187 15232 server_base.cc:1034] running on GCE node
I20260929 02:58:42.808761 15232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:42.809698 15232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:42.810907 15232 hybrid_clock.cc:648] HybridClock initialized: now 1790650722810855 us; error 43 us; skew 500 ppm
I20260929 02:58:42.811236 15232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:42 dist-test-slave-m1rv krb5kdc[15126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650722, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:42.814234 15232 init.cc:377] Logged in from keytab as oryx/127.8.208.65@KRBTEST.COM (short username oryx)
I20260929 02:58:42.815611 15232 webserver.cc:460] Webserver started at http://127.8.208.65:34391/ using document root <none> and password file <none>
I20260929 02:58:42.816284 15232 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:42.816435 15232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:42.816740 15232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:42.819702 15232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "bfa9940c93b24f89b616b77e3b7d5e98"
format_stamp: "Formatted at 2026-09-29 02:58:42 on dist-test-slave-m1rv"
server_key: "8aab21a9b451a1aa990a1c83dcaca6f6"
server_key_iv: "68e38157e9c21e6571571a0bb9761dba"
server_key_version: "encryptionkey@0"
I20260929 02:58:42.820428 15232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "bfa9940c93b24f89b616b77e3b7d5e98"
format_stamp: "Formatted at 2026-09-29 02:58:42 on dist-test-slave-m1rv"
server_key: "8aab21a9b451a1aa990a1c83dcaca6f6"
server_key_iv: "68e38157e9c21e6571571a0bb9761dba"
server_key_version: "encryptionkey@0"
I20260929 02:58:42.825276 15232 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 02:58:42.828451 15247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:42.829715 15232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 02:58:42.829881 15232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "bfa9940c93b24f89b616b77e3b7d5e98"
format_stamp: "Formatted at 2026-09-29 02:58:42 on dist-test-slave-m1rv"
server_key: "8aab21a9b451a1aa990a1c83dcaca6f6"
server_key_iv: "68e38157e9c21e6571571a0bb9761dba"
server_key_version: "encryptionkey@0"
I20260929 02:58:42.830072 15232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:42.859299 15232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:42.863381 15232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:42.863608 15232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:42.864428 15232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:42.865655 15232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:42.865738 15232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:42.865851 15232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:42.865904 15232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:42.898708 15232 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:39239
I20260929 02:58:42.898752 15359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:39239 every 8 connection(s)
I20260929 02:58:42.899995 15232 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:58:42.902998 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 15232
I20260929 02:58:42.903198 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:58:42.904049 9025 external_mini_cluster.cc:1428] Setting key a0810b839e7b8b80b32036a9f6868cdc
Sep 29 02:58:42 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650722, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
I20260929 02:58:42.919997 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:42.902583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:57921 (local address 127.8.208.126:33867)
0929 02:58:42.902998 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:42.903002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:42.904410 (+ 1408us) server_negotiation.cc:407] Connection header received
0929 02:58:42.906598 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:42.906604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:42.906695 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:42.906808 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:42.908233 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:42.908914 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:42.910504 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:42.910899 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:42.911029 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:42.914601 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:42.914606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:42.914609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:42.914639 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:42.917045 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:42.917776 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:42.917786 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:42.917788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:42.917906 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:42.918326 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:42.918331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:42.918334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:42.918589 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:42.918738 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:42.919402 (+ 664us) server_negotiation.cc:299] Negotiation successful
0929 02:58:42.919615 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":227,"threads_started":1}
I20260929 02:58:42.920974 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:42.903310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.65:57921)
0929 02:58:42.904077 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:42.904149 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:42.906348 (+ 2199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:42.907051 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:42.907059 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:42.907532 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:42.908074 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:42.908091 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:42.909065 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:42.909070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:42.910355 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:42.910366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:42.911110 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:42.911118 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:42.911353 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:42.912050 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:42.912071 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:42.914398 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:42.917219 (+ 2821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:42.917227 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:42.917244 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:42.917651 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:42.918016 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:42.918021 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:42.918025 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:42.918210 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:42.918723 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:42.918731 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:42.919059 (+ 328us) client_negotiation.cc:769] Sending connection context
0929 02:58:42.919309 (+ 250us) client_negotiation.cc:240] Negotiation successful
0929 02:58:42.919825 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":359,"threads_started":1}
WARNING: no policy specified for oryx/127.8.208.66@KRBTEST.COM; defaulting to no policy
I20260929 02:58:42.922580 15360 heartbeater.cc:347] Connected to a master server at 127.8.208.126:33867
I20260929 02:58:42.922865 15360 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:42.923547 15360 heartbeater.cc:511] Master 127.8.208.126:33867 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.66@KRBTEST.COM" created.
I20260929 02:58:42.925977 15171 ts_manager.cc:194] Registered new tserver with Master: bfa9940c93b24f89b616b77e3b7d5e98 (127.8.208.65:39239)
I20260929 02:58:42.927405 15171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:57921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab.
Entry for principal oryx/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab.
I20260929 02:58:42.950603 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:33867
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab
--principal=oryx/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:43.266153 15366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:43.266444 15366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:43.266547 15366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:43.273540 15366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:43.273643 15366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:43.273751 15366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:58:43.283285 15366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab
--principal=oryx/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:33867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:43.284525 15366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:43.285785 15366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:43.298668 15371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:43.298696 15372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:43.298635 15374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:43.299537 15366 server_base.cc:1034] running on GCE node
I20260929 02:58:43.300100 15366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:43.301033 15366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:43.302255 15366 hybrid_clock.cc:648] HybridClock initialized: now 1790650723302201 us; error 67 us; skew 500 ppm
I20260929 02:58:43.302574 15366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:43 dist-test-slave-m1rv krb5kdc[15126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650723, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:43.305812 15366 init.cc:377] Logged in from keytab as oryx/127.8.208.66@KRBTEST.COM (short username oryx)
I20260929 02:58:43.307345 15366 webserver.cc:460] Webserver started at http://127.8.208.66:38399/ using document root <none> and password file <none>
I20260929 02:58:43.308012 15366 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:43.308135 15366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:43.308434 15366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:43.311308 15366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "fac6550165ad4ba98559f94c8870bc46"
format_stamp: "Formatted at 2026-09-29 02:58:43 on dist-test-slave-m1rv"
server_key: "d921d246d7f66bab38ac900b233b02c7"
server_key_iv: "eb3441a2671350e5f328c1ea43978a03"
server_key_version: "encryptionkey@0"
I20260929 02:58:43.312055 15366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "fac6550165ad4ba98559f94c8870bc46"
format_stamp: "Formatted at 2026-09-29 02:58:43 on dist-test-slave-m1rv"
server_key: "d921d246d7f66bab38ac900b233b02c7"
server_key_iv: "eb3441a2671350e5f328c1ea43978a03"
server_key_version: "encryptionkey@0"
I20260929 02:58:43.317461 15366 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260929 02:58:43.320775 15381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:43.322095 15366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:58:43.322264 15366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "fac6550165ad4ba98559f94c8870bc46"
format_stamp: "Formatted at 2026-09-29 02:58:43 on dist-test-slave-m1rv"
server_key: "d921d246d7f66bab38ac900b233b02c7"
server_key_iv: "eb3441a2671350e5f328c1ea43978a03"
server_key_version: "encryptionkey@0"
I20260929 02:58:43.322466 15366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:43.342916 15366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:43.347370 15366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:43.347658 15366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:43.348582 15366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:43.349926 15366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:43.350030 15366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:43.350171 15366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:43.350243 15366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:43.383244 15366 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:46003
I20260929 02:58:43.383330 15493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:46003 every 8 connection(s)
I20260929 02:58:43.384521 15366 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:58:43.389081 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 15366
I20260929 02:58:43.389278 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:58:43.389838 9025 external_mini_cluster.cc:1428] Setting key f30bf86cfddc41811286ba21091128ed
Sep 29 02:58:43 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650723, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
I20260929 02:58:43.407501 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:43.386994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:46147 (local address 127.8.208.126:33867)
0929 02:58:43.387164 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:43.387168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:43.388224 (+ 1056us) server_negotiation.cc:407] Connection header received
0929 02:58:43.390662 (+ 2438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:43.390667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:43.390760 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:43.390866 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:43.392519 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:43.393274 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:43.395638 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:43.396026 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:43.396117 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:43.400281 (+ 4164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:43.400290 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:43.400299 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:43.400346 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:43.404200 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:43.404918 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:43.404927 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:43.404933 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:43.405091 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:43.405551 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:43.405558 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:43.405564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:43.405963 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:43.406120 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:43.406774 (+ 654us) server_negotiation.cc:299] Negotiation successful
0929 02:58:43.407042 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:58:43.408149 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:43.387404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.66:46147)
0929 02:58:43.388068 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:43.388103 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:43.390404 (+ 2301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:43.391093 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:43.391100 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:43.391589 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:43.392344 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:43.392368 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:43.393420 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:43.393428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:43.395501 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:43.395518 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:43.396246 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:43.396255 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:43.396584 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:43.397375 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:43.397414 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:43.400044 (+ 2630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:43.404338 (+ 4294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:43.404345 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:43.404359 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:43.404759 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:43.405248 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:43.405251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:43.405253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:43.405440 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:43.406082 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:43.406088 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:43.406398 (+ 310us) client_negotiation.cc:769] Sending connection context
0929 02:58:43.406622 (+ 224us) client_negotiation.cc:240] Negotiation successful
0929 02:58:43.406991 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":255,"threads_started":1}
I20260929 02:58:43.409713 15494 heartbeater.cc:347] Connected to a master server at 127.8.208.126:33867
I20260929 02:58:43.410013 15494 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:43.410692 15494 heartbeater.cc:511] Master 127.8.208.126:33867 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.8.208.67@KRBTEST.COM; defaulting to no policy
I20260929 02:58:43.412214 15171 ts_manager.cc:194] Registered new tserver with Master: fac6550165ad4ba98559f94c8870bc46 (127.8.208.66:46003)
I20260929 02:58:43.413094 15171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:46147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab.
Entry for principal oryx/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab.
I20260929 02:58:43.440048 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:33867
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab
--principal=oryx/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:43.754226 15499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:43.754541 15499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:43.754705 15499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:43.762310 15499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:43.762461 15499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:43.762640 15499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:58:43.772473 15499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab
--principal=oryx/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:33867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:43.773844 15499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:43.775489 15499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:43.789005 15507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:43.789026 15504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:43.789026 15505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:43.789742 15499 server_base.cc:1034] running on GCE node
I20260929 02:58:43.790547 15499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:43.791555 15499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:43.792833 15499 hybrid_clock.cc:648] HybridClock initialized: now 1790650723792796 us; error 61 us; skew 500 ppm
I20260929 02:58:43.793156 15499 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:43 dist-test-slave-m1rv krb5kdc[15126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650723, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:43.796402 15499 init.cc:377] Logged in from keytab as oryx/127.8.208.67@KRBTEST.COM (short username oryx)
I20260929 02:58:43.797976 15499 webserver.cc:460] Webserver started at http://127.8.208.67:45935/ using document root <none> and password file <none>
I20260929 02:58:43.798648 15499 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:43.798789 15499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:43.799081 15499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:43.802029 15499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "521bef1b580742b8b758d014adc3d92d"
format_stamp: "Formatted at 2026-09-29 02:58:43 on dist-test-slave-m1rv"
server_key: "91652acb5e9f63d25571d2519a746846"
server_key_iv: "e0c5257ff1c4354085f121f30b185931"
server_key_version: "encryptionkey@0"
I20260929 02:58:43.802775 15499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "521bef1b580742b8b758d014adc3d92d"
format_stamp: "Formatted at 2026-09-29 02:58:43 on dist-test-slave-m1rv"
server_key: "91652acb5e9f63d25571d2519a746846"
server_key_iv: "e0c5257ff1c4354085f121f30b185931"
server_key_version: "encryptionkey@0"
I20260929 02:58:43.808012 15499 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:58:43.811345 15514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:43.812577 15499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 02:58:43.812803 15499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "521bef1b580742b8b758d014adc3d92d"
format_stamp: "Formatted at 2026-09-29 02:58:43 on dist-test-slave-m1rv"
server_key: "91652acb5e9f63d25571d2519a746846"
server_key_iv: "e0c5257ff1c4354085f121f30b185931"
server_key_version: "encryptionkey@0"
I20260929 02:58:43.813088 15499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:43.830974 15499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:43.834939 15499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:43.835233 15499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:43.836319 15499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:43.837743 15499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:43.837857 15499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:43.838009 15499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:43.838083 15499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:43.871255 15499 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:46209
I20260929 02:58:43.871344 15626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:46209 every 8 connection(s)
I20260929 02:58:43.873020 15499 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:58:43.879370 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 15499
I20260929 02:58:43.879567 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:58:43.880043 9025 external_mini_cluster.cc:1428] Setting key bb4f00e174b549f87f5bf87bb05e426c
Sep 29 02:58:43 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650723, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
I20260929 02:58:43.895988 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:43.877423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:51111 (local address 127.8.208.126:33867)
0929 02:58:43.879423 (+ 2000us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:43.879432 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:43.879464 (+ 32us) server_negotiation.cc:407] Connection header received
0929 02:58:43.881366 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:43.881373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:43.881467 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:43.881552 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:43.883790 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:43.884522 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:43.886277 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:43.886761 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:43.886858 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:43.890355 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:43.890359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:43.890363 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:43.890394 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:43.892910 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:43.893623 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:43.893628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:43.893631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:43.893779 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:43.894182 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:43.894187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:43.894190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:43.894580 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:43.894734 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:43.895552 (+ 818us) server_negotiation.cc:299] Negotiation successful
0929 02:58:43.895736 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1807}
I20260929 02:58:43.896957 15629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:43.877251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.67:51111)
0929 02:58:43.878170 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:43.878220 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:43.880995 (+ 2775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:43.882110 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:43.882122 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:43.882944 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:43.883621 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:43.883639 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:43.884736 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:43.884741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:43.886141 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:43.886153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:43.886983 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:43.886988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:43.887191 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:43.887837 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:43.887864 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:43.890177 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:43.893029 (+ 2852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:43.893035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:43.893046 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:43.893498 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:43.893892 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:43.893896 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:43.893899 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:43.894087 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:43.894710 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:43.894718 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:43.895119 (+ 401us) client_negotiation.cc:769] Sending connection context
0929 02:58:43.895441 (+ 322us) client_negotiation.cc:240] Negotiation successful
0929 02:58:43.895909 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":388,"threads_started":1}
I20260929 02:58:43.898339 15627 heartbeater.cc:347] Connected to a master server at 127.8.208.126:33867
I20260929 02:58:43.898649 15627 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:43.899523 15627 heartbeater.cc:511] Master 127.8.208.126:33867 requested a full tablet report, sending...
I20260929 02:58:43.900946 15171 ts_manager.cc:194] Registered new tserver with Master: 521bef1b580742b8b758d014adc3d92d (127.8.208.67:46209)
I20260929 02:58:43.901806 15171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:51111
I20260929 02:58:43.907603 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.208.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.125@KRBTEST.COM" created.
I20260929 02:58:43.930855 15360 heartbeater.cc:503] Master 127.8.208.126:33867 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.125.keytab.
Entry for principal oryx/127.8.208.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.125.keytab.
I20260929 02:58:43.952827 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.125:38021
--webserver_interface=127.8.208.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.125.keytab
--principal=oryx/127.8.208.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:33867,127.8.208.125:38021
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:44.271278 15632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:44.271561 15632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:44.271677 15632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:44.278851 15632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:44.278951 15632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:44.279065 15632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:44.279170 15632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:44.279250 15632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:44.288686 15632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:33867,127.8.208.125:38021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.125.keytab
--principal=oryx/127.8.208.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.125:38021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.208.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
W20260929 02:58:44.289021 15632 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.208.126:33867,127.8.208.125:38021'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 02:58:44.290005 15632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:44.291290 15632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:44.302699 15638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:44.302737 15637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:44.302793 15640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:44.305476 15632 server_base.cc:1034] running on GCE node
I20260929 02:58:44.306273 15632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:44.307613 15632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:44.308840 15632 hybrid_clock.cc:648] HybridClock initialized: now 1790650724308779 us; error 42 us; skew 500 ppm
I20260929 02:58:44.309150 15632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:44 dist-test-slave-m1rv krb5kdc[15126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650724, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:44.312520 15632 init.cc:377] Logged in from keytab as oryx/127.8.208.125@KRBTEST.COM (short username oryx)
I20260929 02:58:44.313935 15632 webserver.cc:460] Webserver started at http://127.8.208.125:37147/ using document root <none> and password file <none>
I20260929 02:58:44.314565 15632 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:44.314648 15632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:44.314937 15632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:44.318002 15632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data/instance:
uuid: "f5bff55efd8448438781b118662b2742"
format_stamp: "Formatted at 2026-09-29 02:58:44 on dist-test-slave-m1rv"
server_key: "6d7cada4050f50ff8358aed417fc3c15"
server_key_iv: "4d6708e53b8412713de718acdb3a72c2"
server_key_version: "encryptionkey@0"
I20260929 02:58:44.318686 15632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/wal/instance:
uuid: "f5bff55efd8448438781b118662b2742"
format_stamp: "Formatted at 2026-09-29 02:58:44 on dist-test-slave-m1rv"
server_key: "6d7cada4050f50ff8358aed417fc3c15"
server_key_iv: "4d6708e53b8412713de718acdb3a72c2"
server_key_version: "encryptionkey@0"
I20260929 02:58:44.323618 15632 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 02:58:44.326979 15647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:44.328330 15632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:58:44.328469 15632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/wal
uuid: "f5bff55efd8448438781b118662b2742"
format_stamp: "Formatted at 2026-09-29 02:58:44 on dist-test-slave-m1rv"
server_key: "6d7cada4050f50ff8358aed417fc3c15"
server_key_iv: "4d6708e53b8412713de718acdb3a72c2"
server_key_version: "encryptionkey@0"
I20260929 02:58:44.328665 15632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:44.352161 15632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:44.356141 15632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:44.356355 15632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:44.374132 15632 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.125:38021
I20260929 02:58:44.374151 15698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.125:38021 every 8 connection(s)
I20260929 02:58:44.375427 15632 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
I20260929 02:58:44.378912 15699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:44.380781 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 15632
I20260929 02:58:44.387705 15699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:58:44 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650724, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.125@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
Sep 29 02:58:44 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.208.125@KRBTEST.COM
I20260929 02:58:44.407696 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.390902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.8.208.126:33867)
0929 02:58:44.391059 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:44.391063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:44.391817 (+ 754us) server_negotiation.cc:407] Connection header received
0929 02:58:44.392689 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:44.392694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:44.392772 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:44.392897 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:44.394243 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.394992 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.396802 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.397451 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.397586 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.401209 (+ 3623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:44.401218 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:44.401223 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:44.401263 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:44.404209 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.405155 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.405160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.405163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.405343 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.405840 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.405843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.405845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.406191 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:44.406327 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:44.407130 (+ 803us) server_negotiation.cc:299] Negotiation successful
0929 02:58:44.407348 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:58:44.408732 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.390951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:35604)
0929 02:58:44.391657 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:44.391694 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:44.392462 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:44.392974 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:44.392980 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:44.393734 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:44.394078 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.394094 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.395123 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.395128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.396663 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.396674 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.397671 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.397678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.397902 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.398716 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:44.398737 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:44.400975 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:44.404346 (+ 3371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:44.404356 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:44.404368 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:44.405022 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:44.405475 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:44.405478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:44.405480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:44.405727 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:44.406332 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:44.406337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:44.406773 (+ 436us) client_negotiation.cc:769] Sending connection context
0929 02:58:44.407009 (+ 236us) client_negotiation.cc:240] Negotiation successful
0929 02:58:44.407514 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":1,"thread_start_us":357,"threads_started":1}
I20260929 02:58:44.408749 15702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.383352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.8.208.125:38021)
0929 02:58:44.390073 (+ 6721us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:44.390083 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:44.390121 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:58:44.392449 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:44.392468 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:44.392698 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:44.392903 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:44.394330 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.395826 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.397130 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.397852 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.397982 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.402510 (+ 4528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:44.402518 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:44.402529 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:44.402555 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:44.405460 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.406171 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.406179 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.406182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.406317 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.406694 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.406697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.406699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.407034 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:44.407328 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:44.407653 (+ 325us) server_negotiation.cc:299] Negotiation successful
0929 02:58:44.407904 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":6493,"thread_start_us":6225,"threads_started":1}
I20260929 02:58:44.410722 15699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:58:44.415788 15494 heartbeater.cc:503] Master 127.8.208.126:33867 was elected leader, sending a full tablet report...
Sep 29 02:58:44 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650724, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.125@KRBTEST.COM for oryx/127.8.208.125@KRBTEST.COM
I20260929 02:58:44.422474 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.411394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:38021 (local address 127.0.0.1:50074)
0929 02:58:44.411584 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:44.411601 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:44.411689 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:44.411938 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:44.411941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:44.412219 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:44.412336 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.412342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.413303 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.413306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.414139 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.414148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.414782 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.414785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.414916 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.415469 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:44.415485 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:44.417754 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:44.420281 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:44.420284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:44.420287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:44.420622 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:44.420925 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:44.420928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:44.420930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:44.421049 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:44.421578 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:44.421587 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:44.421699 (+ 112us) client_negotiation.cc:769] Sending connection context
0929 02:58:44.421834 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 02:58:44.422109 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 02:58:44.422535 15702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.411592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50074 (local address 127.8.208.125:38021)
0929 02:58:44.411727 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:44.411731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:44.411744 (+ 13us) server_negotiation.cc:407] Connection header received
0929 02:58:44.411778 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:44.411781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:44.411827 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:44.411899 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:44.412445 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.413092 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.414258 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.414631 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.414744 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.417955 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:44.417959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:44.417962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:44.417988 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:44.420169 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.420712 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.420716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.420718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.420832 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.421134 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.421137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.421139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.421455 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:44.421577 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:44.421932 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 02:58:44.422240 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":50}
I20260929 02:58:44.427114 15699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: Bootstrap starting.
I20260929 02:58:44.430136 15699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:44.431062 15699 log.cc:824] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:44.433795 15699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: No bootstrap required, opened a new log
I20260929 02:58:44.441109 15699 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } }
I20260929 02:58:44.441491 15699 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:44.441567 15699 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5bff55efd8448438781b118662b2742, State: Initialized, Role: FOLLOWER
I20260929 02:58:44.442142 15699 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } }
I20260929 02:58:44.442957 15705 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } } }
I20260929 02:58:44.443302 15705 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:58:44.443910 15699 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 02:58:44.452943 15716 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:58:44.453097 15716 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:58:44.455161 15632 master_runner.cc:428] Detected that this master f5bff55efd8448438781b118662b2742 is joining an existing cluster
I20260929 02:58:44.455271 15632 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.208.125:38021
I20260929 02:58:44.476274 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.458959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35608 (local address 127.8.208.126:33867)
0929 02:58:44.459161 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:44.459167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:44.459813 (+ 646us) server_negotiation.cc:407] Connection header received
0929 02:58:44.460370 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:44.460376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:44.460475 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:44.460549 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:44.462236 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.463405 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.465563 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.466207 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.466308 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.469578 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:44.469584 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:44.469588 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:44.469625 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:44.473402 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.474238 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.474246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.474250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.474394 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.474773 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.474777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.474781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.475067 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:44.475221 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:44.475695 (+ 474us) server_negotiation.cc:299] Negotiation successful
0929 02:58:44.475901 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:58:44.476402 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.459026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:35608)
0929 02:58:44.459715 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:44.459734 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:44.460186 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:44.460758 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:44.460763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:44.461576 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:44.462079 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.462094 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.463576 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.463583 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.465404 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.465425 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.466627 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.466641 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.466872 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.467746 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:44.467777 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:44.469363 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:44.473547 (+ 4184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:44.473553 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:44.473560 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:44.474095 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:44.474511 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:44.474516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:44.474520 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:44.474655 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:44.475188 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:44.475192 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:44.475381 (+ 189us) client_negotiation.cc:769] Sending connection context
0929 02:58:44.475538 (+ 157us) client_negotiation.cc:240] Negotiation successful
0929 02:58:44.475872 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":155,"spinlock_wait_cycles":48896,"thread_start_us":375,"threads_started":1}
I20260929 02:58:44.476472 15702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.460040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.8.208.125:38021)
0929 02:58:44.460177 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:44.460182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:44.460572 (+ 390us) server_negotiation.cc:407] Connection header received
0929 02:58:44.460747 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:44.460752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:44.460814 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:44.460917 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:44.462356 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.463405 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.465549 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.466282 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.466355 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.469566 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:44.469572 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:44.469576 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:44.469609 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:44.473622 (+ 4013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.474277 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.474287 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.474290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.474424 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.474835 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.474840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.474843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.475189 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:44.475324 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:44.475825 (+ 501us) server_negotiation.cc:299] Negotiation successful
0929 02:58:44.476039 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 02:58:44.477269 15723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.459855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:38021 (local address 127.0.0.1:50086)
0929 02:58:44.460477 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:44.460490 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:44.460630 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:44.460938 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:44.460942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:44.461742 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:44.462189 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.462202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.463620 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.463630 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.465404 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.465418 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.466660 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.466665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.466872 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.467838 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:44.467866 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:44.469363 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:44.473733 (+ 4370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:44.473740 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:44.473744 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:44.474185 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:44.474553 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:44.474557 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:44.474561 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:44.474717 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:44.475400 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:44.475404 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:44.475539 (+ 135us) client_negotiation.cc:769] Sending connection context
0929 02:58:44.476540 (+ 1001us) client_negotiation.cc:240] Negotiation successful
0929 02:58:44.476810 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":126,"thread_start_us":340,"threads_started":1}
WARNING: no policy specified for oryx/127.8.208.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.124@KRBTEST.COM" created.
Sep 29 02:58:44 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650722, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.126@KRBTEST.COM for oryx/127.8.208.125@KRBTEST.COM
I20260929 02:58:44.501942 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.482892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:38021 (local address 127.0.0.1:50098)
0929 02:58:44.485743 (+ 2851us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:44.485793 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:44.486020 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:44.486579 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:44.486587 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:44.487411 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:44.487694 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.487713 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.489120 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.489125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.490858 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.490874 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.491945 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.491951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.492136 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.492813 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:44.492830 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:44.495091 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:44.498781 (+ 3690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:44.498795 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:44.498804 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:44.499380 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:44.499821 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:44.499826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:44.499830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:44.500004 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:44.500674 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:44.500683 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:44.500885 (+ 202us) client_negotiation.cc:769] Sending connection context
0929 02:58:44.501100 (+ 215us) client_negotiation.cc:240] Negotiation successful
0929 02:58:44.501502 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2618,"thread_start_us":2487,"threads_started":1}
I20260929 02:58:44.501966 15702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.482909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50098 (local address 127.8.208.125:38021)
0929 02:58:44.483095 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:44.483101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:44.485877 (+ 2776us) server_negotiation.cc:407] Connection header received
0929 02:58:44.486259 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:44.486264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:44.486392 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:44.486509 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:44.487888 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.488985 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.491025 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.491771 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.491914 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.495276 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:44.495281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:44.495284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:44.495313 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:44.498623 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.499524 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.499535 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.499541 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.499697 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.500125 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.500132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.500142 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.500532 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:44.500674 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:44.501256 (+ 582us) server_negotiation.cc:299] Negotiation successful
0929 02:58:44.501512 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:58:44.504297 15171 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.208.125:38021: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f5bff55efd8448438781b118662b2742"
member_type: NON_VOTER
last_known_addr {
host: "127.8.208.125"
port: 38021
}
attrs {
promote: true
}
}
dest_uuid: "802fa531790e45aab9bdccc54d19ae50"
cas_config_opid_index: -1
I20260929 02:58:44.505524 15171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } }
I20260929 02:58:44.514432 15728 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f5bff55efd8448438781b118662b2742 (127.8.208.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } }
I20260929 02:58:44.515564 15674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:44.519852 15728 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.208.125:38021
I20260929 02:58:44.520560 15674 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 FOLLOWER]: Refusing update from remote peer 802fa531790e45aab9bdccc54d19ae50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 02:58:44.520444 15729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } }
I20260929 02:58:44.520848 15632 master.cc:537] Master@127.8.208.125:38021 shutting down...
I20260929 02:58:44.520897 15729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.124.keytab.
Entry for principal oryx/127.8.208.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.124.keytab.
I20260929 02:58:44.525156 15729 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:44.526326 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.124:45209
--webserver_interface=127.8.208.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.124.keytab
--principal=oryx/127.8.208.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:33867,127.8.208.125:38021,127.8.208.124:45209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:44.529320 15156 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260929 02:58:44.535543 15156 consensus_peers.cc:487] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 -> Peer f5bff55efd8448438781b118662b2742 (127.8.208.125:38021): Couldn't send request to peer f5bff55efd8448438781b118662b2742. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 02:58:44.535763 15632 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:44.536211 15632 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:44.536404 15632 tablet_replica.cc:330] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: stopping tablet replica
I20260929 02:58:44.553800 15632 master.cc:552] Master@127.8.208.125:38021 shutdown complete.
I20260929 02:58:44.553902 15632 master_runner.cc:305] Clearing existing system tablet
I20260929 02:58:44.554447 15632 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:44.554558 15632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:44.557669 15632 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260929 02:58:44.559334 15734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:44.559689 15632 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:44.559948 15632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 02:58:44.560058 15632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/wal
uuid: "f5bff55efd8448438781b118662b2742"
format_stamp: "Formatted at 2026-09-29 02:58:44 on dist-test-slave-m1rv"
server_key: "6d7cada4050f50ff8358aed417fc3c15"
server_key_iv: "4d6708e53b8412713de718acdb3a72c2"
server_key_version: "encryptionkey@0"
I20260929 02:58:44.564996 15632 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:44.571821 15632 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 02:58:44.571992 15632 log.cc:1196] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 02:58:44.572418 15632 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: Deleting consensus metadata
I20260929 02:58:44.572675 15632 master_runner.cc:315] Copying system tablet from 127.8.208.126:33867
I20260929 02:58:44.575104 15632 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: tablet copy: Beginning tablet copy session from remote peer at address 127.8.208.126:33867
I20260929 02:58:44.596705 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.575946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35622 (local address 127.8.208.126:33867)
0929 02:58:44.576133 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:44.576139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:44.581778 (+ 5639us) server_negotiation.cc:407] Connection header received
0929 02:58:44.581958 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:44.581964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:44.582068 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:44.582189 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:44.583267 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.584621 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.586924 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.587575 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.587710 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.590022 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:44.590029 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:44.590035 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:44.590077 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:44.593681 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.594464 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.594470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.594475 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.594623 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:44.595099 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:44.595104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:44.595108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:44.595495 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:44.595647 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:44.596094 (+ 447us) server_negotiation.cc:299] Negotiation successful
0929 02:58:44.596329 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:58:44.596750 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.575866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:35622)
0929 02:58:44.581614 (+ 5748us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:44.581641 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:44.581811 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:44.582230 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:44.582235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:44.582757 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:44.583062 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.583079 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.584843 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.584850 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.586736 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.586756 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.587760 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.587765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.587938 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.588718 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:44.588742 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:44.589803 (+ 1061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:44.593826 (+ 4023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:44.593831 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:44.593835 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:44.594319 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:44.594746 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:44.594750 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:44.594754 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:44.594967 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:44.595624 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:44.595630 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:44.595815 (+ 185us) client_negotiation.cc:769] Sending connection context
0929 02:58:44.595963 (+ 148us) client_negotiation.cc:240] Negotiation successful
0929 02:58:44.596291 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5600,"thread_start_us":5467,"threads_started":1}
I20260929 02:58:44.597986 15191 tablet_copy_service.cc:140] P 802fa531790e45aab9bdccc54d19ae50: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f5bff55efd8448438781b118662b2742 ({username='oryx', principal='oryx/127.8.208.125@KRBTEST.COM'} at 127.0.0.1:35622)
I20260929 02:58:44.598256 15191 tablet_copy_service.cc:161] P 802fa531790e45aab9bdccc54d19ae50: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f5bff55efd8448438781b118662b2742 at {username='oryx', principal='oryx/127.8.208.125@KRBTEST.COM'} at 127.0.0.1:35622: session id = f5bff55efd8448438781b118662b2742-00000000000000000000000000000000
I20260929 02:58:44.602881 15191 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 02:58:44.605033 15632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 02:58:44.611958 15632 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: tablet copy: Starting download of 0 data blocks...
I20260929 02:58:44.612274 15632 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: tablet copy: Starting download of 1 WAL segments...
I20260929 02:58:44.615274 15632 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 02:58:44.619733 15191 tablet_copy_service.cc:342] P 802fa531790e45aab9bdccc54d19ae50: Request end of tablet copy session f5bff55efd8448438781b118662b2742-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.208.125@KRBTEST.COM'} at 127.0.0.1:35622
I20260929 02:58:44.619952 15191 tablet_copy_service.cc:433] P 802fa531790e45aab9bdccc54d19ae50: ending tablet copy session f5bff55efd8448438781b118662b2742-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f5bff55efd8448438781b118662b2742
I20260929 02:58:44.631672 15632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:44.634341 15741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:44.635591 15742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:44.642495 15744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:44.646579 15632 server_base.cc:1034] running on GCE node
I20260929 02:58:44.646905 15632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:44.647653 15632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:44.648859 15632 hybrid_clock.cc:648] HybridClock initialized: now 1790650724648800 us; error 34 us; skew 500 ppm
I20260929 02:58:44.649082 15632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:44 dist-test-slave-m1rv krb5kdc[15126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650724, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:44.651311 15632 init.cc:377] Logged in from keytab as oryx/127.8.208.125@KRBTEST.COM (short username oryx)
I20260929 02:58:44.654048 15632 webserver.cc:460] Webserver started at http://127.8.208.125:34439/ using document root <none> and password file <none>
I20260929 02:58:44.654415 15632 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:44.654498 15632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:44.657754 15632 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260929 02:58:44.659673 15751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:44.660181 15632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 02:58:44.660368 15632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/wal
uuid: "f5bff55efd8448438781b118662b2742"
format_stamp: "Formatted at 2026-09-29 02:58:44 on dist-test-slave-m1rv"
server_key: "6d7cada4050f50ff8358aed417fc3c15"
server_key_iv: "4d6708e53b8412713de718acdb3a72c2"
server_key_version: "encryptionkey@0"
I20260929 02:58:44.660583 15632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:44.677670 15632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:44.681879 15632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:44.707844 15632 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.125:38021
I20260929 02:58:44.707913 15802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.125:38021 every 8 connection(s)
I20260929 02:58:44.708952 15632 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
I20260929 02:58:44.714370 15803 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:58:44.719276 15803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: Bootstrap starting.
I20260929 02:58:44.736107 15803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:58:44.736677 15803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: Bootstrap complete.
I20260929 02:58:44.737744 15803 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } }
I20260929 02:58:44.738003 15803 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f5bff55efd8448438781b118662b2742, State: Initialized, Role: LEARNER
I20260929 02:58:44.738298 15803 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } }
I20260929 02:58:44.739161 15806 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } }
I20260929 02:58:44.739490 15803 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:44.739629 15806 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: This master's current role is: LEARNER
I20260929 02:58:44.751921 15817 catalog_manager.cc:1259] Loaded cluster ID: b7690fbfec114c13b738f2f439821c9c
I20260929 02:58:44.752059 15817 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: loading cluster ID for follower catalog manager: success
I20260929 02:58:44.755231 15817 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: acquiring CA information for follower catalog manager: success
I20260929 02:58:44.757453 15817 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 02:58:44.897617 15732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:44.897943 15732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:44.898155 15732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260929 02:58:44.904556 15627 heartbeater.cc:503] Master 127.8.208.126:33867 was elected leader, sending a full tablet report...
W20260929 02:58:44.905941 15732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:44.906064 15732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:44.906155 15732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:44.906268 15732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:44.906342 15732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:44.917209 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.909212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:38021 (local address 127.0.0.1:50108)
0929 02:58:44.909443 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:44.909462 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:44.909566 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:44.910542 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:44.910547 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:44.910578 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:44.910773 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.910783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.912550 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.912556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.914570 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:44.914597 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.916284 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.916291 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:44.916517 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.916637 (+ 120us) client_negotiation.cc:769] Sending connection context
0929 02:58:44.916730 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 02:58:44.916872 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260929 02:58:44.917313 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:44.909542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50108 (local address 127.8.208.125:38021)
0929 02:58:44.910168 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:44.910174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:44.910196 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:58:44.910259 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:44.910264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:44.910337 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:44.910460 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:44.910966 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.912395 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.914760 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:44.916099 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:44.916215 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:44.916463 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:44.916769 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 02:58:44.916906 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":318,"threads_started":1}
I20260929 02:58:44.918725 15778 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260929 02:58:44.918797 15732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:33867,127.8.208.125:38021,127.8.208.124:45209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.124.keytab
--principal=oryx/127.8.208.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.124:45209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.208.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:44.919286 15806 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 802fa531790e45aab9bdccc54d19ae50. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } }
I20260929 02:58:44.919546 15806 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: This master's current role is: LEARNER
I20260929 02:58:44.920231 15732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:44.921547 15732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:44.933525 15823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:44.933547 15824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:44.934536 15826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:44.934974 15732 server_base.cc:1034] running on GCE node
I20260929 02:58:44.935705 15732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:44.937070 15732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:44.938294 15732 hybrid_clock.cc:648] HybridClock initialized: now 1790650724938255 us; error 71 us; skew 500 ppm
I20260929 02:58:44.938632 15732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:44 dist-test-slave-m1rv krb5kdc[15126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650724, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:44.946892 15732 init.cc:377] Logged in from keytab as oryx/127.8.208.124@KRBTEST.COM (short username oryx)
I20260929 02:58:44.948432 15732 webserver.cc:460] Webserver started at http://127.8.208.124:43929/ using document root <none> and password file <none>
I20260929 02:58:44.949316 15732 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:44.949456 15732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:44.949831 15732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:44.952950 15732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data/instance:
uuid: "2dc0f606d3ab453c8935c02a5f5671a8"
format_stamp: "Formatted at 2026-09-29 02:58:44 on dist-test-slave-m1rv"
server_key: "c3944563cb20d1929b556cc313c06d86"
server_key_iv: "8c36b251d278ca952ad64cb91ec5789a"
server_key_version: "encryptionkey@0"
I20260929 02:58:44.953824 15732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/wal/instance:
uuid: "2dc0f606d3ab453c8935c02a5f5671a8"
format_stamp: "Formatted at 2026-09-29 02:58:44 on dist-test-slave-m1rv"
server_key: "c3944563cb20d1929b556cc313c06d86"
server_key_iv: "8c36b251d278ca952ad64cb91ec5789a"
server_key_version: "encryptionkey@0"
I20260929 02:58:44.959105 15732 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:58:44.962638 15833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:44.964144 15732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260929 02:58:44.964321 15732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/wal
uuid: "2dc0f606d3ab453c8935c02a5f5671a8"
format_stamp: "Formatted at 2026-09-29 02:58:44 on dist-test-slave-m1rv"
server_key: "c3944563cb20d1929b556cc313c06d86"
server_key_iv: "8c36b251d278ca952ad64cb91ec5789a"
server_key_version: "encryptionkey@0"
I20260929 02:58:44.964526 15732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:44.981487 15732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:44.985283 15732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:44.985571 15732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:45.003985 15732 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.124:45209
I20260929 02:58:45.004009 15884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.124:45209 every 8 connection(s)
I20260929 02:58:45.005440 15732 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
I20260929 02:58:45.009346 15885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:45.012800 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 15732
I20260929 02:58:45.017649 15885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:58:45 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.208.124@KRBTEST.COM
Sep 29 02:58:45 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650724, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.124@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
I20260929 02:58:45.035596 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.020592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35636 (local address 127.8.208.126:33867)
0929 02:58:45.020787 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.020795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.021504 (+ 709us) server_negotiation.cc:407] Connection header received
0929 02:58:45.021675 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.021679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.021737 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.021844 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.023193 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.023884 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.025582 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.025963 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.026096 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.029643 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.029648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.029652 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.029683 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.032492 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.033255 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.033259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.033261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.033379 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.033891 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.033896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.033899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.034260 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.034415 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.034982 (+ 567us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.035215 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:58:45.035965 15889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.020503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:35636)
0929 02:58:45.021361 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:45.021405 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:45.021562 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:45.021899 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:45.021906 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:45.022734 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:45.023045 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.023054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.024028 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.024034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.025413 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.025431 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.026176 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.026183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.026342 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.026936 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:45.026956 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:45.029457 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:45.032625 (+ 3168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.032637 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.032644 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.033067 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.033493 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.033500 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.033505 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.033700 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.034406 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:45.034416 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:45.034601 (+ 185us) client_negotiation.cc:769] Sending connection context
0929 02:58:45.034849 (+ 248us) client_negotiation.cc:240] Negotiation successful
0929 02:58:45.035215 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":404,"threads_started":1}
I20260929 02:58:45.035993 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.015238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53088 (local address 127.8.208.124:45209)
0929 02:58:45.016104 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.016121 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.016179 (+ 58us) server_negotiation.cc:407] Connection header received
0929 02:58:45.017947 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.017969 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.018378 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.018750 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.020306 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.022404 (+ 2098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.023820 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.024494 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.024690 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.029263 (+ 4573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.029272 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.029284 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.029316 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.031871 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.032516 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.032523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.032538 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.032687 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.033072 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.033078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.033081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.033619 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.033997 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.034373 (+ 376us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.034813 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":400,"threads_started":1}
I20260929 02:58:45.037523 15885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:58:45 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650724, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.124@KRBTEST.COM for oryx/127.8.208.125@KRBTEST.COM
I20260929 02:58:45.050521 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.038237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50124 (local address 127.8.208.125:38021)
0929 02:58:45.038411 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.038417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.038437 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:58:45.038494 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.038499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.038576 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.038684 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.039385 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.040204 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.041404 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.041931 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.042028 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.045067 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.045071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.045075 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.045106 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.047948 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.048581 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.048586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.048590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.048729 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.049100 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.049106 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.049110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.049435 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.049590 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.049974 (+ 384us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.050186 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:58:45.050521 15889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.038143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:38021 (local address 127.0.0.1:50124)
0929 02:58:45.038304 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:45.038323 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:45.038410 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:45.038721 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:45.038728 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:45.039102 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:45.039268 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.039276 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.040329 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.040333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.041279 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.041290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.042047 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.042051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.042164 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.042639 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:45.042654 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:45.044866 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:45.048071 (+ 3205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.048078 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.048082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.048460 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.048849 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.048856 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.048860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.048995 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.049556 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:45.049561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:45.049712 (+ 151us) client_negotiation.cc:769] Sending connection context
0929 02:58:45.049855 (+ 143us) client_negotiation.cc:240] Negotiation successful
0929 02:58:45.050133 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 02:58:45.051707 15885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:58:45 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650724, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.124@KRBTEST.COM for oryx/127.8.208.124@KRBTEST.COM
I20260929 02:58:45.062700 15889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.052338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:45209 (local address 127.0.0.1:53090)
0929 02:58:45.052549 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:45.052563 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:45.052632 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:45.052868 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:45.052871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:45.053209 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:45.053352 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.053360 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.054242 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.054245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.055037 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.055046 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.055629 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.055632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.055763 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.056212 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:45.056226 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:45.058074 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:45.060552 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.060556 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.060558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.060925 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.061317 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.061320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.061322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.061462 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.061942 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:45.061963 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:45.062084 (+ 121us) client_negotiation.cc:769] Sending connection context
0929 02:58:45.062184 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 02:58:45.062427 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 02:58:45.062939 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.052437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.8.208.124:45209)
0929 02:58:45.052587 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.052591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.052606 (+ 15us) server_negotiation.cc:407] Connection header received
0929 02:58:45.052716 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.052719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.052766 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.052838 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.053482 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.054142 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.055143 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.055510 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.055602 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.058232 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.058237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.058240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.058266 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.060445 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.061018 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.061022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.061024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.061133 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.061550 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.061553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.061555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.061850 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.061976 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.062265 (+ 289us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.062580 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":51}
I20260929 02:58:45.067395 15885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: Bootstrap starting.
I20260929 02:58:45.070528 15885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:45.071470 15885 log.cc:824] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:45.073866 15885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: No bootstrap required, opened a new log
I20260929 02:58:45.080526 15885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } }
I20260929 02:58:45.080807 15885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:45.080891 15885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc0f606d3ab453c8935c02a5f5671a8, State: Initialized, Role: FOLLOWER
I20260929 02:58:45.081421 15885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } }
I20260929 02:58:45.082168 15891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } } }
I20260929 02:58:45.082506 15891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:58:45.083094 15885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:45.090772 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 15232
W20260929 02:58:45.092965 15902 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:58:45.093284 15902 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:58:45.096473 15732 master_runner.cc:428] Detected that this master 2dc0f606d3ab453c8935c02a5f5671a8 is joining an existing cluster
I20260929 02:58:45.096597 15732 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.208.124:45209
Sep 29 02:58:45 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.208.65@KRBTEST.COM
I20260929 02:58:45.116747 15904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.092494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39996 (local address 127.8.208.65:39239)
0929 02:58:45.093933 (+ 1439us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.093940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.093984 (+ 44us) server_negotiation.cc:407] Connection header received
0929 02:58:45.094038 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.094043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.094229 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.094409 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.096014 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.097300 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.099405 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.099962 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.100112 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.107050 (+ 6938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.107071 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.107093 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.107150 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.112483 (+ 5333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.113531 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.113541 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.113547 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.113703 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.114186 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.114191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.114194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.114698 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.115014 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.115709 (+ 695us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.116053 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1294,"thread_start_us":1132,"threads_started":1}
I20260929 02:58:45.116946 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.100811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35642 (local address 127.8.208.126:33867)
0929 02:58:45.100992 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.100998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.101660 (+ 662us) server_negotiation.cc:407] Connection header received
0929 02:58:45.101903 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.101909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.101985 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.102051 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.103462 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.104609 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.106732 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.107444 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.107549 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.110163 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.110171 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.110178 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.110231 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.113682 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.114538 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.114543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.114547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.114703 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.115175 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.115180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.115184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.115526 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.115664 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.116356 (+ 692us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.116610 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:58:45.118876 15909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.100860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:35642)
0929 02:58:45.101562 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:45.101580 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:45.101760 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:45.102253 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:45.102258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:45.102901 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:45.103292 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.103304 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.104812 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.104823 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.106549 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.106564 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.107826 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.107833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.107997 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.108711 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:45.108743 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:45.109951 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:45.113857 (+ 3906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.113868 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.113873 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.114400 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.114850 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.114856 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.114860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.115042 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.115749 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:45.115758 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:45.116001 (+ 243us) client_negotiation.cc:769] Sending connection context
0929 02:58:45.117317 (+ 1316us) client_negotiation.cc:240] Negotiation successful
0929 02:58:45.118103 (+ 786us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":392,"threads_started":1}
I20260929 02:58:45.119656 15355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:45.133878 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.102226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50136 (local address 127.8.208.125:38021)
0929 02:58:45.102408 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.102415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.113519 (+ 11104us) server_negotiation.cc:407] Connection header received
0929 02:58:45.115027 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.115035 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.115122 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.115189 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.118366 (+ 3177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.119398 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.121516 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.122201 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.122299 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.124788 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.124795 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.124799 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.124839 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.130330 (+ 5491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.131183 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.131191 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.131196 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.131374 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.131895 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.131901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.131906 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.132287 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.132412 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.133113 (+ 701us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.133440 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:58:45.135749 15910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.101742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:38021 (local address 127.0.0.1:50136)
0929 02:58:45.113422 (+ 11680us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:45.113442 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:45.114728 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:45.115418 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:45.115423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:45.116879 (+ 1456us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:45.118177 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.118191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.119574 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.119579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.121297 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.121310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.122609 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.122616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.122779 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.123466 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:45.123493 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:45.124605 (+ 1112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:45.130511 (+ 5906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.130517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.130521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.131012 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.131545 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.131551 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.131556 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.131766 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.132525 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:45.132530 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:45.132789 (+ 259us) client_negotiation.cc:769] Sending connection context
0929 02:58:45.134093 (+ 1304us) client_negotiation.cc:240] Negotiation successful
0929 02:58:45.135247 (+ 1154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11538,"thread_start_us":11788,"threads_started":1}
I20260929 02:58:45.139910 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.114072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53098 (local address 127.8.208.124:45209)
0929 02:58:45.114280 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.114285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.118748 (+ 4463us) server_negotiation.cc:407] Connection header received
0929 02:58:45.120642 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.120647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.120704 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.120760 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.122221 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.123312 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.129277 (+ 5965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.129966 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.130053 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.132998 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.133007 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.133014 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.133056 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.136894 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.137762 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.137767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.137770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.137894 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.138288 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.138293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.138296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.138644 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.138795 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.139303 (+ 508us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.139574 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:58:45.141229 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.114392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:45209 (local address 127.0.0.1:53098)
0929 02:58:45.118670 (+ 4278us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:45.118686 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:45.120495 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:45.120989 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:45.121013 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:45.121832 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:45.122081 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.122093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.123536 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.123540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.125023 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.125034 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.130340 (+ 5306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.130349 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.130595 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.131377 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:45.131405 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:45.132783 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:45.137067 (+ 4284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.137072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.137077 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.137647 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.138036 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.138042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.138046 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.138182 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.138860 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:45.138865 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:45.139010 (+ 145us) client_negotiation.cc:769] Sending connection context
0929 02:58:45.140394 (+ 1384us) client_negotiation.cc:240] Negotiation successful
0929 02:58:45.140746 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4110,"thread_start_us":3711,"threads_started":1}
Sep 29 02:58:45 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650722, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.126@KRBTEST.COM for oryx/127.8.208.124@KRBTEST.COM
I20260929 02:58:45.151952 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.129471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:45209 (local address 127.0.0.1:53106)
0929 02:58:45.129796 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:45.129814 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:45.129917 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:45.135377 (+ 5460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:45.135384 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:45.135828 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:45.136055 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.136067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.138259 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.138265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.139744 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.139759 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.140931 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.140938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.141119 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.141797 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:45.141822 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:45.144853 (+ 3031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:45.148994 (+ 4141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.149000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.149005 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.149521 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.149943 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.149948 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.149952 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.150106 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.150842 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:45.150848 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:45.151016 (+ 168us) client_negotiation.cc:769] Sending connection context
0929 02:58:45.151196 (+ 180us) client_negotiation.cc:240] Negotiation successful
0929 02:58:45.151546 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260929 02:58:45.152046 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.130139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53106 (local address 127.8.208.124:45209)
0929 02:58:45.134919 (+ 4780us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.134924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.134949 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:58:45.135059 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.135066 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.135160 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.136304 (+ 1144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.136687 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.138104 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.140050 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.140726 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.141488 (+ 762us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.145076 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.145083 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.145088 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.145122 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.148842 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.149645 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.149651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.149655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.149809 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.150260 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.150265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.150269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.150690 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.150820 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.151308 (+ 488us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.151538 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4612,"thread_start_us":4421,"threads_started":1}
I20260929 02:58:45.153957 15171 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.208.124:45209: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8"
member_type: NON_VOTER
last_known_addr {
host: "127.8.208.124"
port: 45209
}
attrs {
promote: true
}
}
dest_uuid: "802fa531790e45aab9bdccc54d19ae50"
cas_config_opid_index: 5
I20260929 02:58:45.155004 15171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } }
I20260929 02:58:45.158008 15778 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 LEARNER]: Refusing update from remote peer 802fa531790e45aab9bdccc54d19ae50: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 02:58:45.159221 15860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:45.163228 15860 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 FOLLOWER]: Refusing update from remote peer 802fa531790e45aab9bdccc54d19ae50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 02:58:45.156819 15729 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2dc0f606d3ab453c8935c02a5f5671a8 (127.8.208.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } } }
I20260929 02:58:45.166586 15914 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:45.177474 15729 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.208.124:45209
I20260929 02:58:45.177590 15916 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 02:58:45.178212 15732 master.cc:537] Master@127.8.208.124:45209 shutting down...
I20260929 02:58:45.179268 15729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } } }
I20260929 02:58:45.179647 15729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:45.182641 15860 raft_consensus.cc:858] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } }
W20260929 02:58:45.183626 15860 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } }
I20260929 02:58:45.190755 15778 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2dc0f606d3ab453c8935c02a5f5671a8 (127.8.208.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } } }
I20260929 02:58:45.193982 15806 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } } }
I20260929 02:58:45.194284 15806 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: This master's current role is: LEARNER
I20260929 02:58:45.194763 15729 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50: attempting to promote NON_VOTER f5bff55efd8448438781b118662b2742 to VOTER
I20260929 02:58:45.195434 15729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } }
I20260929 02:58:45.197496 15778 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 LEARNER]: Refusing update from remote peer 802fa531790e45aab9bdccc54d19ae50: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260929 02:58:45.198030 15916 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260929 02:58:45.200107 15729 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f5bff55efd8448438781b118662b2742 (127.8.208.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } } }
I20260929 02:58:45.200807 15778 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f5bff55efd8448438781b118662b2742 (127.8.208.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } } }
I20260929 02:58:45.203848 15729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } } }
I20260929 02:58:45.204191 15729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:45.204041 15915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } } }
I20260929 02:58:45.204437 15915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:45.205471 15806 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } } }
I20260929 02:58:45.205698 15806 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 02:58:45.226281 15156 consensus_peers.cc:487] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 -> Peer 2dc0f606d3ab453c8935c02a5f5671a8 (127.8.208.124:45209): Couldn't send request to peer 2dc0f606d3ab453c8935c02a5f5671a8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 02:58:45.233880 15860 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f5bff55efd8448438781b118662b2742 (127.8.208.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } }
I20260929 02:58:45.237459 15925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } }
I20260929 02:58:45.237998 15925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260929 02:58:45.238462 15891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 802fa531790e45aab9bdccc54d19ae50. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } }
I20260929 02:58:45.238670 15891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260929 02:58:45.238814 15921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: NON_VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: true } } }
I20260929 02:58:45.239125 15921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260929 02:58:45.241530 15732 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260929 02:58:45.241936 15732 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260929 02:58:45.242116 15732 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: stopping tablet replica
I20260929 02:58:45.249240 15732 master.cc:552] Master@127.8.208.124:45209 shutdown complete.
I20260929 02:58:45.249408 15732 master_runner.cc:305] Clearing existing system tablet
I20260929 02:58:45.249910 15732 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:45.250056 15732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:45.253262 15732 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260929 02:58:45.255040 15927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:45.255417 15732 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:45.255831 15732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:58:45.255973 15732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/wal
uuid: "2dc0f606d3ab453c8935c02a5f5671a8"
format_stamp: "Formatted at 2026-09-29 02:58:44 on dist-test-slave-m1rv"
server_key: "c3944563cb20d1929b556cc313c06d86"
server_key_iv: "8c36b251d278ca952ad64cb91ec5789a"
server_key_version: "encryptionkey@0"
I20260929 02:58:45.260758 15732 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:45.261431 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 15232
I20260929 02:58:45.267752 15732 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 02:58:45.267968 15732 log.cc:1196] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 02:58:45.268517 15732 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: Deleting consensus metadata
I20260929 02:58:45.268814 15732 master_runner.cc:315] Copying system tablet from 127.8.208.126:33867
I20260929 02:58:45.271489 15732 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: tablet copy: Beginning tablet copy session from remote peer at address 127.8.208.126:33867
I20260929 02:58:45.284152 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:39239
--local_ip_for_outbound_sockets=127.8.208.65
--tserver_master_addrs=127.8.208.126:33867
--webserver_port=34391
--webserver_interface=127.8.208.65
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab
--principal=oryx/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.208.126:33867,127.8.208.125:38021
--tserver_master_addrs=127.8.208.126:33867,127.8.208.125:38021,127.8.208.124:45209 with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:58:45.297492 15933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.272384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:35644)
0929 02:58:45.281673 (+ 9289us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:45.281695 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:45.281871 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:45.282326 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:45.282331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:45.283122 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:45.283429 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.283442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.285035 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.285041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.286683 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.286698 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.287705 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.287711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.287929 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.288712 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:45.288741 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:45.289859 (+ 1118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:45.294232 (+ 4373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.294240 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.294247 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.294823 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.295247 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.295255 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.295261 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.295466 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.296050 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:45.296058 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:45.296310 (+ 252us) client_negotiation.cc:769] Sending connection context
0929 02:58:45.296477 (+ 167us) client_negotiation.cc:240] Negotiation successful
0929 02:58:45.296860 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9123,"thread_start_us":8979,"threads_started":1}
I20260929 02:58:45.298339 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.272362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35644 (local address 127.8.208.126:33867)
0929 02:58:45.272594 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.272600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.281837 (+ 9237us) server_negotiation.cc:407] Connection header received
0929 02:58:45.282014 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.282023 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.282152 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.282276 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.283606 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.284886 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.286838 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.287546 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.287682 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.290079 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.290087 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.290092 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.290137 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.294088 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.294963 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.294969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.294972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.295125 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.295592 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.295598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.295601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.295916 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.296065 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.297435 (+ 1370us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.297986 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260929 02:58:45.298799 15191 tablet_copy_service.cc:140] P 802fa531790e45aab9bdccc54d19ae50: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2dc0f606d3ab453c8935c02a5f5671a8 ({username='oryx', principal='oryx/127.8.208.124@KRBTEST.COM'} at 127.0.0.1:35644)
I20260929 02:58:45.298960 15191 tablet_copy_service.cc:161] P 802fa531790e45aab9bdccc54d19ae50: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2dc0f606d3ab453c8935c02a5f5671a8 at {username='oryx', principal='oryx/127.8.208.124@KRBTEST.COM'} at 127.0.0.1:35644: session id = 2dc0f606d3ab453c8935c02a5f5671a8-00000000000000000000000000000000
I20260929 02:58:45.302595 15191 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 02:58:45.304456 15732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 02:58:45.311038 15732 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: tablet copy: Starting download of 0 data blocks...
I20260929 02:58:45.311321 15732 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: tablet copy: Starting download of 1 WAL segments...
I20260929 02:58:45.313562 15732 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 02:58:45.317844 15191 tablet_copy_service.cc:342] P 802fa531790e45aab9bdccc54d19ae50: Request end of tablet copy session 2dc0f606d3ab453c8935c02a5f5671a8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.208.124@KRBTEST.COM'} at 127.0.0.1:35644
I20260929 02:58:45.317999 15191 tablet_copy_service.cc:433] P 802fa531790e45aab9bdccc54d19ae50: ending tablet copy session 2dc0f606d3ab453c8935c02a5f5671a8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2dc0f606d3ab453c8935c02a5f5671a8
I20260929 02:58:45.330596 15732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:45.333370 15935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:45.338886 15936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:45.344043 15732 server_base.cc:1034] running on GCE node
W20260929 02:58:45.344403 15938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:45.344859 15732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:45.345690 15732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:45.346896 15732 hybrid_clock.cc:648] HybridClock initialized: now 1790650725346856 us; error 29 us; skew 500 ppm
I20260929 02:58:45.347175 15732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:45 dist-test-slave-m1rv krb5kdc[15126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650725, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:45.349222 15732 init.cc:377] Logged in from keytab as oryx/127.8.208.124@KRBTEST.COM (short username oryx)
I20260929 02:58:45.350620 15732 webserver.cc:460] Webserver started at http://127.8.208.124:35703/ using document root <none> and password file <none>
I20260929 02:58:45.350930 15732 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:45.351038 15732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:45.353853 15732 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260929 02:58:45.355504 15945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:45.355919 15732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260929 02:58:45.356055 15732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/wal
uuid: "2dc0f606d3ab453c8935c02a5f5671a8"
format_stamp: "Formatted at 2026-09-29 02:58:44 on dist-test-slave-m1rv"
server_key: "c3944563cb20d1929b556cc313c06d86"
server_key_iv: "8c36b251d278ca952ad64cb91ec5789a"
server_key_version: "encryptionkey@0"
I20260929 02:58:45.356237 15732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:45.368264 15732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:45.371750 15732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:45.401118 15732 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.124:45209
I20260929 02:58:45.401147 15996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.124:45209 every 8 connection(s)
I20260929 02:58:45.402134 15732 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
I20260929 02:58:45.409086 15997 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:58:45.414535 15997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: Bootstrap starting.
I20260929 02:58:45.431828 15997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:58:45.432331 15997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: Bootstrap complete.
I20260929 02:58:45.433251 15997 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } }
I20260929 02:58:45.433521 15997 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2dc0f606d3ab453c8935c02a5f5671a8, State: Initialized, Role: LEARNER
I20260929 02:58:45.433769 15997 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } }
I20260929 02:58:45.435566 16000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } } }
I20260929 02:58:45.435889 15997 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:45.436026 16000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: This master's current role is: LEARNER
I20260929 02:58:45.461360 16011 catalog_manager.cc:1259] Loaded cluster ID: b7690fbfec114c13b738f2f439821c9c
I20260929 02:58:45.461477 16011 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: loading cluster ID for follower catalog manager: success
I20260929 02:58:45.464272 16011 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: acquiring CA information for follower catalog manager: success
I20260929 02:58:45.466085 16011 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 02:58:45.650389 16012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.643688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.8.208.124:45209)
0929 02:58:45.644227 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.644232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.644245 (+ 13us) server_negotiation.cc:407] Connection header received
0929 02:58:45.644301 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.644305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.644365 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.644471 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:45.644961 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.646089 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.648495 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.649326 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.649425 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.649655 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.649996 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.650092 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":288,"threads_started":1}
I20260929 02:58:45.650452 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.643514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:45209 (local address 127.0.0.1:53122)
0929 02:58:45.643690 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:45.643711 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:45.643825 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:45.644541 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:45.644550 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:45.644586 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:45.644790 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.644804 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.646267 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.646276 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.648346 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.648365 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.649562 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.649570 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.649815 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.649860 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 02:58:45.649980 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 02:58:45.650128 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 02:58:45.651402 15972 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260929 02:58:45.651885 16000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 802fa531790e45aab9bdccc54d19ae50. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } } }
I20260929 02:58:45.652145 16000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: This master's current role is: LEARNER
W20260929 02:58:45.702811 15934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:45.703069 15934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:45.703161 15934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:45.710625 15934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:45.710721 15934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:45.710887 15934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:58:45.720587 15934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab
--principal=oryx/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:39239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=34391
--tserver_master_addrs=127.8.208.126:33867,127.8.208.125:38021,127.8.208.124:45209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:45.721887 15934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:45.723135 15934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:45.736286 16020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:45.736320 16018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:45.736356 16017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:45.741035 15934 server_base.cc:1034] running on GCE node
I20260929 02:58:45.741678 15934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:45.742590 15934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:45.743786 15934 hybrid_clock.cc:648] HybridClock initialized: now 1790650725743737 us; error 45 us; skew 500 ppm
I20260929 02:58:45.744128 15934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:45 dist-test-slave-m1rv krb5kdc[15126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650725, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:45.747640 15934 init.cc:377] Logged in from keytab as oryx/127.8.208.65@KRBTEST.COM (short username oryx)
I20260929 02:58:45.749094 15934 webserver.cc:460] Webserver started at http://127.8.208.65:34391/ using document root <none> and password file <none>
I20260929 02:58:45.749924 15934 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:45.750064 15934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:45.755219 15934 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 02:58:45.757964 16027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:45.759193 15934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:58:45.759375 15934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "bfa9940c93b24f89b616b77e3b7d5e98"
format_stamp: "Formatted at 2026-09-29 02:58:42 on dist-test-slave-m1rv"
server_key: "8aab21a9b451a1aa990a1c83dcaca6f6"
server_key_iv: "68e38157e9c21e6571571a0bb9761dba"
server_key_version: "encryptionkey@0"
I20260929 02:58:45.760324 15934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:45.786545 15934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:45.790710 15934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:45.790992 15934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:45.791983 15934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:45.793464 15934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:45.793593 15934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:45.793715 15934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:45.793802 15934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:45.826808 15934 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:39239
I20260929 02:58:45.826869 16139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:39239 every 8 connection(s)
I20260929 02:58:45.828423 15934 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:58:45.838308 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 15934
I20260929 02:58:45.838727 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 15366
Sep 29 02:58:45 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650725, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
Sep 29 02:58:45 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650725, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.125@KRBTEST.COM
Sep 29 02:58:45 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650725, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.124@KRBTEST.COM
I20260929 02:58:45.860077 16147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.834087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:52965 (local address 127.8.208.126:33867)
0929 02:58:45.836985 (+ 2898us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.836993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.837018 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:58:45.837090 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.837098 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.837239 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.837370 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.839470 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.840688 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.845045 (+ 4357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.845727 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.845882 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.852138 (+ 6256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.852143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.852146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.852176 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.855573 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.856530 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.856538 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.856542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.856780 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.857316 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.857321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.857324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.857746 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.857881 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.859399 (+ 1518us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.859633 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2755,"thread_start_us":2655,"threads_started":1}
I20260929 02:58:45.861107 16146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.833454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:57705 (local address 127.8.208.125:38021)
0929 02:58:45.834393 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.834400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.834440 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:58:45.836116 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.836123 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.836225 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.836344 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.839980 (+ 3636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.841104 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.844947 (+ 3843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.845578 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.845700 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.853459 (+ 7759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.853466 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.853470 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.853506 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.857933 (+ 4427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.858813 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.858819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.858824 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.858959 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.859365 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.859373 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.859376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.859722 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.859845 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.860511 (+ 666us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.860727 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":815,"thread_start_us":705,"threads_started":1}
I20260929 02:58:45.862172 16012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.831539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:51467 (local address 127.8.208.124:45209)
0929 02:58:45.831801 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.831812 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.833073 (+ 1261us) server_negotiation.cc:407] Connection header received
0929 02:58:45.836029 (+ 2956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.836040 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.836136 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.836225 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.839488 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.840979 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.845864 (+ 4885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.846644 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.846745 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.855425 (+ 8680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.855432 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.855438 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.855485 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.858902 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.859804 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.859810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.859814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.859961 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.860476 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.860481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.860486 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.860855 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.861015 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.861527 (+ 512us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.861810 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 02:58:45.862170 16148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.834367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.65:52965)
0929 02:58:45.834957 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:45.834981 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:45.835660 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:45.837442 (+ 1782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:45.837448 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:45.838068 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:45.839230 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.839271 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.840909 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.840914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.843889 (+ 2975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.843912 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.847085 (+ 3173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.847091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.847230 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.847957 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:45.847986 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:45.851913 (+ 3927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:45.855725 (+ 3812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.855733 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.855750 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.856350 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.856913 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.856918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.856921 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.857193 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.857980 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:45.857988 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:45.858405 (+ 417us) client_negotiation.cc:769] Sending connection context
0929 02:58:45.858752 (+ 347us) client_negotiation.cc:240] Negotiation successful
0929 02:58:45.860573 (+ 1821us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":203,"thread_start_us":292,"threads_started":1}
I20260929 02:58:45.862178 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.833281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:38021 (local address 127.8.208.65:57705)
0929 02:58:45.833950 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:45.833967 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:45.835663 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:45.836666 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:45.836682 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:45.837854 (+ 1172us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:45.839808 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.839822 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.841343 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.841350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.844805 (+ 3455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.844825 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.846370 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.846379 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.846673 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.847536 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:45.847580 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:45.853244 (+ 5664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:45.858093 (+ 4849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.858098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.858102 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.858678 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.859082 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.859087 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.859091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.859233 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.859930 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:45.859935 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:45.860078 (+ 143us) client_negotiation.cc:769] Sending connection context
0929 02:58:45.860232 (+ 154us) client_negotiation.cc:240] Negotiation successful
0929 02:58:45.861532 (+ 1300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":2470,"spinlock_wait_cycles":14208,"thread_start_us":291,"threads_started":1}
I20260929 02:58:45.863616 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.832070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:45209 (local address 127.8.208.65:51467)
0929 02:58:45.832886 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:45.832941 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:45.835660 (+ 2719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:45.836665 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:45.836698 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:45.837864 (+ 1166us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:45.839229 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.839272 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.841364 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.841374 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.845551 (+ 4177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:45.845574 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.847823 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.847837 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:45.849837 (+ 2000us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.850518 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:45.850534 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:45.854938 (+ 4404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:45.859089 (+ 4151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.859094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.859097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.859669 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.860166 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:45.860174 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:45.860180 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:45.860343 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:45.861074 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:45.861079 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:45.861255 (+ 176us) client_negotiation.cc:769] Sending connection context
0929 02:58:45.862398 (+ 1143us) client_negotiation.cc:240] Negotiation successful
0929 02:58:45.863169 (+ 771us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":143,"spinlock_wait_cycles":16384,"thread_start_us":296,"threads_started":1}
I20260929 02:58:45.864045 16141 heartbeater.cc:347] Connected to a master server at 127.8.208.126:33867
I20260929 02:58:45.864207 16142 heartbeater.cc:347] Connected to a master server at 127.8.208.125:38021
I20260929 02:58:45.864421 16142 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:45.865244 16140 heartbeater.cc:347] Connected to a master server at 127.8.208.124:45209
I20260929 02:58:45.865396 16140 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:45.865476 16142 heartbeater.cc:511] Master 127.8.208.125:38021 requested a full tablet report, sending...
I20260929 02:58:45.865916 16140 heartbeater.cc:511] Master 127.8.208.124:45209 requested a full tablet report, sending...
I20260929 02:58:45.864425 16141 heartbeater.cc:464] Registering TS with master...
Sep 29 02:58:45 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.208.66@KRBTEST.COM
I20260929 02:58:45.866662 16141 heartbeater.cc:511] Master 127.8.208.126:33867 requested a full tablet report, sending...
I20260929 02:58:45.867900 15962 ts_manager.cc:194] Registered new tserver with Master: bfa9940c93b24f89b616b77e3b7d5e98 (127.8.208.65:39239)
I20260929 02:58:45.867901 15768 ts_manager.cc:194] Registered new tserver with Master: bfa9940c93b24f89b616b77e3b7d5e98 (127.8.208.65:39239)
I20260929 02:58:45.869268 15171 ts_manager.cc:194] Re-registered known tserver with Master: bfa9940c93b24f89b616b77e3b7d5e98 (127.8.208.65:39239)
I20260929 02:58:45.870441 15171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:52965
I20260929 02:58:45.875435 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:45.839840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44538 (local address 127.8.208.66:46003)
0929 02:58:45.846546 (+ 6706us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:45.846554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:45.853240 (+ 6686us) server_negotiation.cc:407] Connection header received
0929 02:58:45.853421 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:45.853431 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:45.853730 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:45.853920 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:45.855892 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.857568 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.859445 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:45.860566 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:45.860681 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:45.867600 (+ 6919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:45.867611 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:45.867623 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:45.867656 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:45.871430 (+ 3774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.872314 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.872322 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.872325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.872471 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:45.872879 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:45.872883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:45.872886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:45.873853 (+ 967us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:45.874165 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:45.874624 (+ 459us) server_negotiation.cc:299] Negotiation successful
0929 02:58:45.874971 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6537,"thread_start_us":6275,"threads_started":1}
I20260929 02:58:45.876925 15489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:45.960058 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 15366
I20260929 02:58:45.976871 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:46003
--local_ip_for_outbound_sockets=127.8.208.66
--tserver_master_addrs=127.8.208.126:33867
--webserver_port=38399
--webserver_interface=127.8.208.66
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab
--principal=oryx/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.208.126:33867,127.8.208.125:38021
--tserver_master_addrs=127.8.208.126:33867,127.8.208.125:38021,127.8.208.124:45209 with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:58:46.026592 15916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: NON_VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: true } } }
I20260929 02:58:46.027021 15916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:46.028429 15916 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50: attempting to promote NON_VOTER 2dc0f606d3ab453c8935c02a5f5671a8 to VOTER
I20260929 02:58:46.028978 15916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } }
I20260929 02:58:46.030823 15778 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 FOLLOWER]: Refusing update from remote peer 802fa531790e45aab9bdccc54d19ae50: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260929 02:58:46.030843 15972 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 LEARNER]: Refusing update from remote peer 802fa531790e45aab9bdccc54d19ae50: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260929 02:58:46.031380 15916 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260929 02:58:46.031762 16153 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260929 02:58:46.034974 15778 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2dc0f606d3ab453c8935c02a5f5671a8 (127.8.208.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } } }
I20260929 02:58:46.035296 15972 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2dc0f606d3ab453c8935c02a5f5671a8 (127.8.208.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } } }
I20260929 02:58:46.034364 15916 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2dc0f606d3ab453c8935c02a5f5671a8 (127.8.208.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } } }
I20260929 02:58:46.039038 16000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } } }
I20260929 02:58:46.039288 16000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:58:46.041693 16157 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } } }
I20260929 02:58:46.042085 16157 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:58:46.041872 16153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } } }
I20260929 02:58:46.042205 16153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:46.042033 16154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } } }
I20260929 02:58:46.042356 16154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:46.045552 16154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "802fa531790e45aab9bdccc54d19ae50" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } } }
I20260929 02:58:46.045825 16154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50 [sys.catalog]: This master's current role is: LEADER
W20260929 02:58:46.304579 16152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:46.304893 16152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:46.305045 16152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:46.312180 16152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:46.312321 16152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:46.312506 16152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:58:46.322456 16152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab
--principal=oryx/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:46003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=38399
--tserver_master_addrs=127.8.208.126:33867,127.8.208.125:38021,127.8.208.124:45209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:46.323727 16152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:46.324994 16152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:46.337971 16163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:46.338069 16164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:46.338037 16166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:46.338971 16152 server_base.cc:1034] running on GCE node
I20260929 02:58:46.339609 16152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:46.340611 16152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:46.341797 16152 hybrid_clock.cc:648] HybridClock initialized: now 1790650726341739 us; error 59 us; skew 500 ppm
I20260929 02:58:46.342145 16152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:46 dist-test-slave-m1rv krb5kdc[15126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650726, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:46.345412 16152 init.cc:377] Logged in from keytab as oryx/127.8.208.66@KRBTEST.COM (short username oryx)
I20260929 02:58:46.347255 16152 webserver.cc:460] Webserver started at http://127.8.208.66:38399/ using document root <none> and password file <none>
I20260929 02:58:46.347934 16152 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:46.348040 16152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:46.353557 16152 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 02:58:46.356020 16173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:46.357465 16152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:58:46.357604 16152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "fac6550165ad4ba98559f94c8870bc46"
format_stamp: "Formatted at 2026-09-29 02:58:43 on dist-test-slave-m1rv"
server_key: "d921d246d7f66bab38ac900b233b02c7"
server_key_iv: "eb3441a2671350e5f328c1ea43978a03"
server_key_version: "encryptionkey@0"
I20260929 02:58:46.358501 16152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:46.376308 16152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:46.380251 16152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:46.380467 16152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:46.381361 16152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:46.382607 16152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:46.382692 16152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:46.382771 16152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:46.382822 16152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:46.416886 16152 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:46003
I20260929 02:58:46.416956 16285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:46003 every 8 connection(s)
I20260929 02:58:46.418398 16152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:58:46.426298 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 16152
I20260929 02:58:46.426605 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 15499
Sep 29 02:58:46 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650726, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
Sep 29 02:58:46 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650726, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.125@KRBTEST.COM
Sep 29 02:58:46 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650726, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.124@KRBTEST.COM
Sep 29 02:58:46 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.208.67@KRBTEST.COM
I20260929 02:58:46.446030 16293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:46.423082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:41333 (local address 127.8.208.126:33867)
0929 02:58:46.424044 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:46.424050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:46.424070 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:58:46.425113 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:46.425119 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:46.425220 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:46.425359 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:46.427571 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.428716 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.430998 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.431611 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.431743 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:46.436653 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:46.436659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:46.436663 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:46.436710 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:46.440350 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:46.441148 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:46.441153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:46.441157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:46.441348 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:46.442793 (+ 1445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:46.442800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:46.442804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:46.443170 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:46.443315 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:46.445423 (+ 2108us) server_negotiation.cc:299] Negotiation successful
0929 02:58:46.445637 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":746,"threads_started":1}
I20260929 02:58:46.447757 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:46.422926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.66:41333)
0929 02:58:46.423597 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:46.423614 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:46.424870 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:46.425568 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:46.425575 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:46.426629 (+ 1054us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:46.427406 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:46.427425 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.428864 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.428872 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:46.430857 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:46.430868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.431850 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.431857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:46.432082 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:46.432849 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:46.432878 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:46.436395 (+ 3517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:46.440487 (+ 4092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:46.440496 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:46.440512 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:46.441019 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:46.442422 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:46.442428 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:46.442433 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:46.442683 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:46.444480 (+ 1797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:46.444490 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:46.444929 (+ 439us) client_negotiation.cc:769] Sending connection context
0929 02:58:46.445347 (+ 418us) client_negotiation.cc:240] Negotiation successful
0929 02:58:46.445950 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":310,"threads_started":1}
I20260929 02:58:46.448478 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:46.421206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:40757 (local address 127.8.208.124:45209)
0929 02:58:46.421749 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:46.421755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:46.422575 (+ 820us) server_negotiation.cc:407] Connection header received
0929 02:58:46.425129 (+ 2554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:46.425134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:46.425233 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:46.425314 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:46.433052 (+ 7738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.434263 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.435905 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.436547 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.436684 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:46.441593 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:46.441602 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:46.441610 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:46.441655 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:46.445280 (+ 3625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:46.446168 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:46.446176 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:46.446180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:46.446351 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:46.446847 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:46.446852 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:46.446855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:46.447203 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:46.447332 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:46.447879 (+ 547us) server_negotiation.cc:299] Negotiation successful
0929 02:58:46.448090 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":248,"threads_started":1}
I20260929 02:58:46.449291 16287 heartbeater.cc:347] Connected to a master server at 127.8.208.126:33867
I20260929 02:58:46.449594 16287 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:46.450489 16287 heartbeater.cc:511] Master 127.8.208.126:33867 requested a full tablet report, sending...
I20260929 02:58:46.450546 16297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:46.427429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38534 (local address 127.8.208.67:46209)
0929 02:58:46.429901 (+ 2472us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:46.429909 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:46.429936 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:58:46.430000 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:46.430010 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:46.430261 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:46.430422 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:46.431947 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.433473 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.435402 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.436260 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.436711 (+ 451us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:46.443168 (+ 6457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:46.443181 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:46.443201 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:46.443254 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:46.447313 (+ 4059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:46.448248 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:46.448258 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:46.448262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:46.448417 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:46.448962 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:46.448968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:46.448971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:46.449343 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:46.449664 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:46.450003 (+ 339us) server_negotiation.cc:299] Negotiation successful
0929 02:58:46.450182 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2306,"thread_start_us":2149,"threads_started":1}
I20260929 02:58:46.451399 16295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:46.424071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:38021 (local address 127.8.208.66:56991)
0929 02:58:46.424585 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:46.424600 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:46.424877 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:46.425575 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:46.425579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:46.427707 (+ 2128us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:46.427985 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:46.427996 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.431604 (+ 3608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.431608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:46.433242 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:46.433256 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.435047 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.435053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:46.435282 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:46.436972 (+ 1690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:46.436995 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:46.440365 (+ 3370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:46.444407 (+ 4042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:46.444415 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:46.444420 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:46.444967 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:46.446645 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:46.446651 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:46.446655 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:46.446795 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:46.447488 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:46.447493 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:46.447644 (+ 151us) client_negotiation.cc:769] Sending connection context
0929 02:58:46.447802 (+ 158us) client_negotiation.cc:240] Negotiation successful
0929 02:58:46.448936 (+ 1134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":276,"threads_started":1}
I20260929 02:58:46.451656 15622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:46.452267 15171 ts_manager.cc:194] Re-registered known tserver with Master: fac6550165ad4ba98559f94c8870bc46 (127.8.208.66:46003)
I20260929 02:58:46.453496 15171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:41333
I20260929 02:58:46.454852 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:46.421755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:45209 (local address 127.8.208.66:40757)
0929 02:58:46.422429 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:46.422462 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:46.424875 (+ 2413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:46.425569 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:46.425577 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:46.432451 (+ 6874us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:46.432854 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:46.432866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.434423 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.434427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:46.435751 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:46.435767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.436721 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.436727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:46.436920 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:46.437687 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:46.437715 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:46.441303 (+ 3588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:46.445430 (+ 4127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:46.445437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:46.445441 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:46.446031 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:46.446495 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:46.446502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:46.446505 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:46.446721 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:46.447421 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:46.447425 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:46.447589 (+ 164us) client_negotiation.cc:769] Sending connection context
0929 02:58:46.448667 (+ 1078us) client_negotiation.cc:240] Negotiation successful
0929 02:58:46.451702 (+ 3035us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":2712,"thread_start_us":247,"threads_started":1}
I20260929 02:58:46.455865 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:46.423802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:56991 (local address 127.8.208.125:38021)
0929 02:58:46.424276 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:46.424282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:46.424679 (+ 397us) server_negotiation.cc:407] Connection header received
0929 02:58:46.425232 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:46.425239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:46.425326 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:46.425418 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:46.428146 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.431454 (+ 3308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.433407 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.434037 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.434156 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:46.440601 (+ 6445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:46.440608 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:46.440614 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:46.440667 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:46.444274 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:46.445113 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:46.445119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:46.445123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:46.446502 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:46.446939 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:46.446946 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:46.446949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:46.447280 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:46.447421 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:46.454002 (+ 6581us) server_negotiation.cc:299] Negotiation successful
0929 02:58:46.454225 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":417,"threads_started":1}
I20260929 02:58:46.456202 16286 heartbeater.cc:347] Connected to a master server at 127.8.208.124:45209
I20260929 02:58:46.456331 16286 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:46.456769 16286 heartbeater.cc:511] Master 127.8.208.124:45209 requested a full tablet report, sending...
I20260929 02:58:46.456837 16288 heartbeater.cc:347] Connected to a master server at 127.8.208.125:38021
I20260929 02:58:46.456948 16288 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:46.457325 16288 heartbeater.cc:511] Master 127.8.208.125:38021 requested a full tablet report, sending...
I20260929 02:58:46.457939 15962 ts_manager.cc:194] Registered new tserver with Master: fac6550165ad4ba98559f94c8870bc46 (127.8.208.66:46003)
I20260929 02:58:46.458384 15768 ts_manager.cc:194] Registered new tserver with Master: fac6550165ad4ba98559f94c8870bc46 (127.8.208.66:46003)
I20260929 02:58:46.535528 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 15499
I20260929 02:58:46.552167 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:46209
--local_ip_for_outbound_sockets=127.8.208.67
--tserver_master_addrs=127.8.208.126:33867
--webserver_port=45935
--webserver_interface=127.8.208.67
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab
--principal=oryx/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.208.126:33867,127.8.208.125:38021
--tserver_master_addrs=127.8.208.126:33867,127.8.208.125:38021,127.8.208.124:45209 with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:58:46.873318 16141 heartbeater.cc:503] Master 127.8.208.126:33867 was elected leader, sending a full tablet report...
W20260929 02:58:46.878723 16299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:46.879052 16299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:46.879210 16299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:46.886493 16299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:46.886677 16299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:46.886857 16299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:58:46.896405 16299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab
--principal=oryx/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:46209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=45935
--tserver_master_addrs=127.8.208.126:33867,127.8.208.125:38021,127.8.208.124:45209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:46.897856 16299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:46.899134 16299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:46.912225 16304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:46.912155 16305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:46.912158 16307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:46.913516 16299 server_base.cc:1034] running on GCE node
I20260929 02:58:46.914137 16299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:46.915119 16299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:46.916347 16299 hybrid_clock.cc:648] HybridClock initialized: now 1790650726916284 us; error 73 us; skew 500 ppm
I20260929 02:58:46.916676 16299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:46 dist-test-slave-m1rv krb5kdc[15126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650726, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:46.919749 16299 init.cc:377] Logged in from keytab as oryx/127.8.208.67@KRBTEST.COM (short username oryx)
I20260929 02:58:46.921322 16299 webserver.cc:460] Webserver started at http://127.8.208.67:45935/ using document root <none> and password file <none>
I20260929 02:58:46.922037 16299 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:46.922194 16299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:46.927352 16299 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260929 02:58:46.930015 16314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:46.931394 16299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 02:58:46.931581 16299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "521bef1b580742b8b758d014adc3d92d"
format_stamp: "Formatted at 2026-09-29 02:58:43 on dist-test-slave-m1rv"
server_key: "91652acb5e9f63d25571d2519a746846"
server_key_iv: "e0c5257ff1c4354085f121f30b185931"
server_key_version: "encryptionkey@0"
I20260929 02:58:46.932595 16299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:46.945561 16299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:46.949944 16299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:46.950217 16299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:46.951232 16299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:58:46.952596 16299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:46.952706 16299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:46.952838 16299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:46.952914 16299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:46.987039 16299 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:46209
I20260929 02:58:46.987136 16426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:46209 every 8 connection(s)
I20260929 02:58:46.988461 16299 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:58:46.991951 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 16299
Sep 29 02:58:47 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650726, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
Sep 29 02:58:47 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650726, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.124@KRBTEST.COM
I20260929 02:58:47.016580 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:46.991587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:48323 (local address 127.8.208.126:33867)
0929 02:58:46.992103 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:46.992110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:46.993110 (+ 1000us) server_negotiation.cc:407] Connection header received
0929 02:58:46.995467 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:46.995473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:46.995563 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:46.995680 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:46.997166 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.998350 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.999813 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.000457 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.000590 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.006413 (+ 5823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:47.006424 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:47.006432 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:47.006490 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:47.010642 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.011784 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.011793 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.011800 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.012001 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.012965 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.012973 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.012979 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.013443 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:47.013601 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.015704 (+ 2103us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.015968 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":305,"threads_started":1}
I20260929 02:58:47.019932 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:46.992172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.67:48323)
0929 02:58:46.992953 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:46.992989 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:46.995205 (+ 2216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:46.995902 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:46.995909 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:46.996417 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:46.996983 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:46.996999 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:46.998471 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:46.998476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:46.999667 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:46.999678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.000648 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.000656 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.000968 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.001948 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:47.001987 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:47.006083 (+ 4096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:47.010841 (+ 4758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:47.010852 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:47.010874 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:47.011624 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:47.012183 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:47.012190 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:47.012209 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:47.012808 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:47.013727 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:47.013739 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:47.014325 (+ 586us) client_negotiation.cc:769] Sending connection context
0929 02:58:47.017268 (+ 2943us) client_negotiation.cc:240] Negotiation successful
0929 02:58:47.018049 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":327,"threads_started":1}
I20260929 02:58:47.022116 16428 heartbeater.cc:347] Connected to a master server at 127.8.208.126:33867
I20260929 02:58:47.022390 16428 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:47.023422 16428 heartbeater.cc:511] Master 127.8.208.126:33867 requested a full tablet report, sending...
I20260929 02:58:47.025254 15171 ts_manager.cc:194] Re-registered known tserver with Master: 521bef1b580742b8b758d014adc3d92d (127.8.208.67:46209)
I20260929 02:58:47.026859 15171 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:48323
I20260929 02:58:47.027803 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:46.994192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:59369 (local address 127.8.208.124:45209)
0929 02:58:46.994662 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:46.994668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:46.997991 (+ 3323us) server_negotiation.cc:407] Connection header received
0929 02:58:46.998055 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:46.998060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:46.998132 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:46.998222 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:47.004279 (+ 6057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.007368 (+ 3089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.009992 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.010888 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.011034 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.017350 (+ 6316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:47.017357 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:47.017363 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:47.017422 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:47.021475 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.025111 (+ 3636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.025117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.025121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.025287 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.025780 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.025785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.025788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.026129 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:47.026252 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.027163 (+ 911us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.027410 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":225,"threads_started":1}
I20260929 02:58:47.031553 16435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:46.993769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:45209 (local address 127.8.208.67:59369)
0929 02:58:46.997597 (+ 3828us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:46.997616 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:46.997730 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:46.998395 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:46.998403 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:47.000355 (+ 1952us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:47.004041 (+ 3686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.004062 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.007582 (+ 3520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.007591 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.009775 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.009794 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.011887 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.011896 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.012138 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.012988 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:47.013020 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:47.017087 (+ 4067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:47.021665 (+ 4578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:47.021670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:47.021674 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:47.024975 (+ 3301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:47.025489 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:47.025494 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:47.025499 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:47.025653 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:47.026322 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:47.026328 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:47.026489 (+ 161us) client_negotiation.cc:769] Sending connection context
0929 02:58:47.026618 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 02:58:47.026926 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3712,"thread_start_us":3548,"threads_started":1}
I20260929 02:58:47.032461 16427 heartbeater.cc:347] Connected to a master server at 127.8.208.124:45209
I20260929 02:58:47.032637 16427 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:47.033105 16427 heartbeater.cc:511] Master 127.8.208.124:45209 requested a full tablet report, sending...
Sep 29 02:58:47 dist-test-slave-m1rv krb5kdc[15126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650726, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.125@KRBTEST.COM
I20260929 02:58:47.034487 15962 ts_manager.cc:194] Registered new tserver with Master: 521bef1b580742b8b758d014adc3d92d (127.8.208.67:46209)
I20260929 02:58:47.040238 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.009180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35648 (local address 127.8.208.126:33867)
0929 02:58:47.015572 (+ 6392us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.015577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.016274 (+ 697us) server_negotiation.cc:407] Connection header received
0929 02:58:47.016329 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.016334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.016404 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.016517 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:47.019115 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.020377 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.025963 (+ 5586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.026645 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.026728 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.031422 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:47.031429 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:47.031434 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:47.031475 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:47.036982 (+ 5507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.037912 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.037918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.037921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.038064 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.038509 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.038513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.038516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.038858 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:47.039151 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.039577 (+ 426us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.039792 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6258,"thread_start_us":6133,"threads_started":1}
I20260929 02:58:47.043889 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.000841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:38021 (local address 127.8.208.67:55509)
0929 02:58:47.015292 (+ 14451us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:47.015315 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:47.015456 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:47.022321 (+ 6865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:47.022327 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:47.023018 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:47.023315 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.023327 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.027932 (+ 4605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.027938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.029808 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.029825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.031015 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.031024 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.031330 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.032219 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:47.032244 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:47.035088 (+ 2844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:47.039887 (+ 4799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:47.039894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:47.039898 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:47.040489 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:47.040955 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:47.040960 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:47.040963 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:47.041136 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:47.042505 (+ 1369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:47.042512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:47.042681 (+ 169us) client_negotiation.cc:769] Sending connection context
0929 02:58:47.043168 (+ 487us) client_negotiation.cc:240] Negotiation successful
0929 02:58:47.043510 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14312,"thread_start_us":14165,"threads_started":1}
I20260929 02:58:47.044914 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:46.997919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:55509 (local address 127.8.208.125:38021)
0929 02:58:47.003780 (+ 5861us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.003784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.015678 (+ 11894us) server_negotiation.cc:407] Connection header received
0929 02:58:47.015765 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.015772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.015873 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.015978 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:47.023604 (+ 7626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.024777 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.030025 (+ 5248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.030808 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.030950 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.035471 (+ 4521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:47.035479 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:47.035484 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:47.035535 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:47.039729 (+ 4194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.040638 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.040645 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.040649 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.040809 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.041650 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.041656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.041660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.042000 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:47.042162 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.044219 (+ 2057us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.044492 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5739,"thread_start_us":5640,"threads_started":1}
I20260929 02:58:47.046268 16429 heartbeater.cc:347] Connected to a master server at 127.8.208.125:38021
I20260929 02:58:47.046418 16429 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:47.046789 16429 heartbeater.cc:511] Master 127.8.208.125:38021 requested a full tablet report, sending...
I20260929 02:58:47.048007 15768 ts_manager.cc:194] Registered new tserver with Master: 521bef1b580742b8b758d014adc3d92d (127.8.208.67:46209)
I20260929 02:58:47.050176 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.017006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.8.208.125:38021)
0929 02:58:47.025244 (+ 8238us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.025251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.028582 (+ 3331us) server_negotiation.cc:407] Connection header received
0929 02:58:47.029916 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.029922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.029996 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.030070 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:47.033092 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.034342 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.037046 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.037848 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.037946 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.042988 (+ 5042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:47.042997 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:47.043003 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:47.043052 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:47.046797 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.047642 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.047649 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.047652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.047824 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.048286 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.048291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.048294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.048620 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:47.048881 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.049477 (+ 596us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.049692 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8084,"thread_start_us":7410,"threads_started":1}
I20260929 02:58:47.044843 15171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:58:47.054206 15171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:47.055205 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.029046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53126 (local address 127.8.208.124:45209)
0929 02:58:47.029554 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.029559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.034457 (+ 4898us) server_negotiation.cc:407] Connection header received
0929 02:58:47.034519 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.034523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.034594 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.034681 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:47.037418 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.038515 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.041304 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.041908 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.042005 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.049095 (+ 7090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:47.049100 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:47.049103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:47.049135 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:47.052511 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.053260 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.053266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.053269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.053402 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.053764 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.053769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.053773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.054038 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:47.054295 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.054642 (+ 347us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.054861 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400}
I20260929 02:58:47.081347 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.072849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:46209 (local address 127.0.0.1:38536)
0929 02:58:47.073459 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:47.073487 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:47.073618 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:47.075635 (+ 2017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:47.075638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:47.075658 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:47.076029 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.076042 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.077453 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.077460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.078775 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.078785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.080591 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.080595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.080710 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.080733 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:58:47.080841 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 02:58:47.080979 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":334,"threads_started":1}
I20260929 02:58:47.081362 16459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.073347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38536 (local address 127.8.208.67:46209)
0929 02:58:47.074733 (+ 1386us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.074740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.074767 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:58:47.074842 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.074848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.075098 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.075304 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:47.076229 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.077321 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.078929 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.080206 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.080333 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.080487 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.080884 (+ 397us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.081038 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1243,"thread_start_us":1064,"threads_started":1}
I20260929 02:58:47.084810 16362 tablet_service.cc:1467] Processing CreateTablet for tablet 6110845402a84bb191bd31eaa977a6a8 (DEFAULT_TABLE table=test-table [id=5517df5aeb6b4c95a45155322f1a43eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:47.086454 16362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6110845402a84bb191bd31eaa977a6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:47.092572 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.074276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40010 (local address 127.8.208.65:39239)
0929 02:58:47.086032 (+ 11756us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.086040 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.086076 (+ 36us) server_negotiation.cc:407] Connection header received
0929 02:58:47.086153 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.086159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.086339 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.086834 (+ 495us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:47.087048 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.087987 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.089878 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.091483 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.091625 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.091837 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.092045 (+ 208us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.092250 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11592,"thread_start_us":11118,"threads_started":1}
I20260929 02:58:47.092569 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.073755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:39239 (local address 127.0.0.1:40010)
0929 02:58:47.084798 (+ 11043us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:47.084820 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:47.084929 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:47.086545 (+ 1616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:47.086553 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:47.086578 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:47.086760 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.086774 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.088124 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.088132 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.089710 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.089725 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.091655 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.091663 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.091840 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.091876 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 02:58:47.091989 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 02:58:47.092133 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10908,"thread_start_us":12854,"threads_started":1}
I20260929 02:58:47.093143 16458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.073345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:46003 (local address 127.0.0.1:44544)
0929 02:58:47.073981 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:47.074003 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:47.074136 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:47.086663 (+ 12527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:47.086668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:47.086693 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:47.087075 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.087087 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.088974 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.088979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.090896 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.090910 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.092526 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.092532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.092727 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.092768 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 02:58:47.092822 (+ 54us) client_negotiation.cc:240] Negotiation successful
0929 02:58:47.092907 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":1199,"threads_started":1}
I20260929 02:58:47.094264 16461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.073996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44544 (local address 127.8.208.66:46003)
0929 02:58:47.086076 (+ 12080us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.086086 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.086114 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:58:47.086202 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.086209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.086449 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.087272 (+ 823us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:47.087489 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.088825 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.091086 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.092284 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.093750 (+ 1466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.093900 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.093953 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.094066 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11904,"thread_start_us":11797,"threads_started":1}
I20260929 02:58:47.097018 16221 tablet_service.cc:1467] Processing CreateTablet for tablet 6110845402a84bb191bd31eaa977a6a8 (DEFAULT_TABLE table=test-table [id=5517df5aeb6b4c95a45155322f1a43eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:47.097090 16075 tablet_service.cc:1467] Processing CreateTablet for tablet 6110845402a84bb191bd31eaa977a6a8 (DEFAULT_TABLE table=test-table [id=5517df5aeb6b4c95a45155322f1a43eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:47.098433 16075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6110845402a84bb191bd31eaa977a6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:47.098438 16221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6110845402a84bb191bd31eaa977a6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:47.101883 16463 tablet_bootstrap.cc:492] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d: Bootstrap starting.
I20260929 02:58:47.106647 16463 tablet_bootstrap.cc:654] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:47.107928 16463 log.cc:824] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:47.110786 16465 tablet_bootstrap.cc:492] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98: Bootstrap starting.
I20260929 02:58:47.110986 16463 tablet_bootstrap.cc:492] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d: No bootstrap required, opened a new log
I20260929 02:58:47.111253 16463 ts_tablet_manager.cc:1397] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.005s
I20260929 02:58:47.112650 16466 tablet_bootstrap.cc:492] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46: Bootstrap starting.
I20260929 02:58:47.114503 16465 tablet_bootstrap.cc:654] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:47.115710 16465 log.cc:824] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:47.115954 16466 tablet_bootstrap.cc:654] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:47.117208 16466 log.cc:824] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:47.118772 16465 tablet_bootstrap.cc:492] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98: No bootstrap required, opened a new log
I20260929 02:58:47.119051 16465 ts_tablet_manager.cc:1397] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 02:58:47.119720 16466 tablet_bootstrap.cc:492] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46: No bootstrap required, opened a new log
I20260929 02:58:47.119972 16466 ts_tablet_manager.cc:1397] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260929 02:58:47.120545 16463 raft_consensus.cc:348] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } }
I20260929 02:58:47.120970 16463 raft_consensus.cc:374] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:47.121078 16463 raft_consensus.cc:729] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521bef1b580742b8b758d014adc3d92d, State: Initialized, Role: FOLLOWER
I20260929 02:58:47.121732 16463 consensus_queue.cc:260] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } }
I20260929 02:58:47.123428 16463 ts_tablet_manager.cc:1428] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
I20260929 02:58:47.123689 16428 heartbeater.cc:503] Master 127.8.208.126:33867 was elected leader, sending a full tablet report...
I20260929 02:58:47.128983 16465 raft_consensus.cc:348] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } }
I20260929 02:58:47.129411 16465 raft_consensus.cc:374] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:47.129510 16465 raft_consensus.cc:729] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa9940c93b24f89b616b77e3b7d5e98, State: Initialized, Role: FOLLOWER
I20260929 02:58:47.129596 16466 raft_consensus.cc:348] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } }
I20260929 02:58:47.129971 16466 raft_consensus.cc:374] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:47.130066 16466 raft_consensus.cc:729] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fac6550165ad4ba98559f94c8870bc46, State: Initialized, Role: FOLLOWER
I20260929 02:58:47.130093 16465 consensus_queue.cc:260] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } }
I20260929 02:58:47.130746 16466 consensus_queue.cc:260] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } }
I20260929 02:58:47.131821 16465 ts_tablet_manager.cc:1428] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260929 02:58:47.132679 16287 heartbeater.cc:503] Master 127.8.208.126:33867 was elected leader, sending a full tablet report...
I20260929 02:58:47.134595 16466 ts_tablet_manager.cc:1428] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
W20260929 02:58:47.171625 16289 tablet.cc:2369] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:58:47.192137 16471 raft_consensus.cc:482] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:47.192483 16471 raft_consensus.cc:504] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } }
I20260929 02:58:47.194255 16471 leader_election.cc:288] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa9940c93b24f89b616b77e3b7d5e98 (127.8.208.65:39239), 521bef1b580742b8b758d014adc3d92d (127.8.208.67:46209)
I20260929 02:58:47.201351 16459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.194784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:46523 (local address 127.8.208.67:46209)
0929 02:58:47.195073 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.195082 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.195821 (+ 739us) server_negotiation.cc:407] Connection header received
0929 02:58:47.196014 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.196018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.196076 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.196183 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:47.196823 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.197638 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.199559 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.200307 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.200458 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.200570 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.200928 (+ 358us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.201061 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260929 02:58:47.201378 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.194636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:49849 (local address 127.8.208.65:39239)
0929 02:58:47.195100 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.195109 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.195775 (+ 666us) server_negotiation.cc:407] Connection header received
0929 02:58:47.195978 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.195983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.196051 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.196117 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:47.196890 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.197640 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.199594 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.200343 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.200518 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.200588 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.200980 (+ 392us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.201098 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282}
I20260929 02:58:47.201543 16472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.194672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:46209 (local address 127.8.208.66:46523)
0929 02:58:47.195676 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:47.195690 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:47.195849 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:47.196214 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:47.196218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:47.196244 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:47.196656 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.196667 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.197780 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.197786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.199397 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.199408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.200525 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.200531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.200755 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.200793 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 02:58:47.200865 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 02:58:47.201023 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":902,"mutex_wait_us":74,"thread_start_us":592,"threads_started":1}
I20260929 02:58:47.201864 16473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.194456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:39239 (local address 127.8.208.66:49849)
0929 02:58:47.195655 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:47.195673 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:47.195820 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:47.196164 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:47.196170 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:47.196201 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:47.196707 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.196717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.197774 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.197781 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.199412 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:47.199425 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.200587 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.200596 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:47.200794 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.200817 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:58:47.200940 (+ 123us) client_negotiation.cc:240] Negotiation successful
0929 02:58:47.201060 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1063,"mutex_wait_us":179,"thread_start_us":959,"threads_started":1}
I20260929 02:58:47.202569 16382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6110845402a84bb191bd31eaa977a6a8" candidate_uuid: "fac6550165ad4ba98559f94c8870bc46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "521bef1b580742b8b758d014adc3d92d" is_pre_election: true
I20260929 02:58:47.202917 16095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6110845402a84bb191bd31eaa977a6a8" candidate_uuid: "fac6550165ad4ba98559f94c8870bc46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" is_pre_election: true
I20260929 02:58:47.203171 16382 raft_consensus.cc:2454] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fac6550165ad4ba98559f94c8870bc46 in term 0.
I20260929 02:58:47.203444 16095 raft_consensus.cc:2454] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fac6550165ad4ba98559f94c8870bc46 in term 0.
I20260929 02:58:47.203898 16176 leader_election.cc:302] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 521bef1b580742b8b758d014adc3d92d, fac6550165ad4ba98559f94c8870bc46; no voters:
I20260929 02:58:47.204329 16471 raft_consensus.cc:2781] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:47.204493 16471 raft_consensus.cc:482] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:47.204600 16471 raft_consensus.cc:3037] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:47.206756 16471 raft_consensus.cc:504] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } }
I20260929 02:58:47.207314 16471 leader_election.cc:288] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [CANDIDATE]: Term 1 election: Requested vote from peers bfa9940c93b24f89b616b77e3b7d5e98 (127.8.208.65:39239), 521bef1b580742b8b758d014adc3d92d (127.8.208.67:46209)
I20260929 02:58:47.207749 16095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6110845402a84bb191bd31eaa977a6a8" candidate_uuid: "fac6550165ad4ba98559f94c8870bc46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa9940c93b24f89b616b77e3b7d5e98"
I20260929 02:58:47.207798 16382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6110845402a84bb191bd31eaa977a6a8" candidate_uuid: "fac6550165ad4ba98559f94c8870bc46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "521bef1b580742b8b758d014adc3d92d"
I20260929 02:58:47.207973 16095 raft_consensus.cc:3037] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:47.208006 16382 raft_consensus.cc:3037] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:47.210209 16095 raft_consensus.cc:2454] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fac6550165ad4ba98559f94c8870bc46 in term 1.
I20260929 02:58:47.210209 16382 raft_consensus.cc:2454] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fac6550165ad4ba98559f94c8870bc46 in term 1.
I20260929 02:58:47.210618 16176 leader_election.cc:302] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 521bef1b580742b8b758d014adc3d92d, fac6550165ad4ba98559f94c8870bc46; no voters:
I20260929 02:58:47.210911 16471 raft_consensus.cc:2781] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:47.211308 16471 raft_consensus.cc:686] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 1 LEADER]: Becoming Leader. State: Replica: fac6550165ad4ba98559f94c8870bc46, State: Running, Role: LEADER
I20260929 02:58:47.211855 16471 consensus_queue.cc:237] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } }
I20260929 02:58:47.217067 15171 catalog_manager.cc:5495] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 reported cstate change: term changed from 0 to 1, leader changed from <none> to fac6550165ad4ba98559f94c8870bc46 (127.8.208.66). New cstate: current_term: 1 leader_uuid: "fac6550165ad4ba98559f94c8870bc46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } health_report { overall_health: HEALTHY } } }
W20260929 02:58:47.241550 16430 tablet.cc:2369] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:58:47.268924 16461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.264009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44556 (local address 127.8.208.66:46003)
0929 02:58:47.264188 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.264228 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.264249 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:58:47.264353 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.264357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.264439 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.264562 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:47.264895 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.265697 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.266953 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.267438 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.267531 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.267946 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:47.268437 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:47.268540 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.268636 (+ 96us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.268699 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:58:47.276677 16382 raft_consensus.cc:1261] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d [term 1 FOLLOWER]: Refusing update from remote peer fac6550165ad4ba98559f94c8870bc46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:47.276757 16095 raft_consensus.cc:1261] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98 [term 1 FOLLOWER]: Refusing update from remote peer fac6550165ad4ba98559f94c8870bc46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:47.277669 16474 consensus_queue.cc:1035] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:47.277987 16471 consensus_queue.cc:1035] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:47.290248 16477 mvcc.cc:205] Tried to move back new op lower bound from 7334505378916241408 to 7334505378663391232. Current Snapshot: MvccSnapshot[applied={T|T < 7334505378916241408}]
I20260929 02:58:47.306398 15170 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:47.306713 15170 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:47.313762 15170 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 802fa531790e45aab9bdccc54d19ae50: Sending DeleteTablet for 3 replicas of tablet 6110845402a84bb191bd31eaa977a6a8
I20260929 02:58:47.315356 16075 tablet_service.cc:1514] Processing DeleteTablet for tablet 6110845402a84bb191bd31eaa977a6a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:47 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:40010
I20260929 02:58:47.315352 16221 tablet_service.cc:1514] Processing DeleteTablet for tablet 6110845402a84bb191bd31eaa977a6a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:47 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:44544
I20260929 02:58:47.316073 16362 tablet_service.cc:1514] Processing DeleteTablet for tablet 6110845402a84bb191bd31eaa977a6a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:47 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:38536
I20260929 02:58:47.318746 16488 tablet_replica.cc:330] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d: stopping tablet replica
I20260929 02:58:47.319192 16488 raft_consensus.cc:2229] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:47.319687 16488 raft_consensus.cc:2258] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:47.321799 16488 ts_tablet_manager.cc:1905] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:47.328783 16488 ts_tablet_manager.cc:1918] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:47.328974 16488 log.cc:1196] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/6110845402a84bb191bd31eaa977a6a8
I20260929 02:58:47.329546 16488 ts_tablet_manager.cc:1939] T 6110845402a84bb191bd31eaa977a6a8 P 521bef1b580742b8b758d014adc3d92d: Deleting consensus metadata
I20260929 02:58:47.331393 15157 catalog_manager.cc:4841] TS 521bef1b580742b8b758d014adc3d92d (127.8.208.67:46209): tablet 6110845402a84bb191bd31eaa977a6a8 (table test-table [id=5517df5aeb6b4c95a45155322f1a43eb]) successfully deleted
I20260929 02:58:47.336354 16486 tablet_replica.cc:330] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46: stopping tablet replica
I20260929 02:58:47.336858 16486 raft_consensus.cc:2229] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:47.337600 16486 raft_consensus.cc:2258] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:47.338217 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 15139
I20260929 02:58:47.339342 15201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:47.339525 16486 ts_tablet_manager.cc:1905] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:47.340029 16487 tablet_replica.cc:330] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98: stopping tablet replica
I20260929 02:58:47.340519 16487 raft_consensus.cc:2229] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:47.342940 16487 raft_consensus.cc:2258] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:47.346925 16487 ts_tablet_manager.cc:1905] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:47.347481 16486 ts_tablet_manager.cc:1918] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:47.347675 16486 log.cc:1196] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/6110845402a84bb191bd31eaa977a6a8
I20260929 02:58:47.348297 16486 ts_tablet_manager.cc:1939] T 6110845402a84bb191bd31eaa977a6a8 P fac6550165ad4ba98559f94c8870bc46: Deleting consensus metadata
I20260929 02:58:47.357241 16487 ts_tablet_manager.cc:1918] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:47.357396 16487 log.cc:1196] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/6110845402a84bb191bd31eaa977a6a8
I20260929 02:58:47.358132 16487 ts_tablet_manager.cc:1939] T 6110845402a84bb191bd31eaa977a6a8 P bfa9940c93b24f89b616b77e3b7d5e98: Deleting consensus metadata
I20260929 02:58:47.363876 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.317712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.8.208.125:38021)
0929 02:58:47.317888 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.317895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.337932 (+ 20037us) server_negotiation.cc:407] Connection header received
0929 02:58:47.341327 (+ 3395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.341336 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.341458 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.341549 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:47.347489 (+ 5940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.348647 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.350692 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.351353 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.351481 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.356710 (+ 5229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:47.356716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:47.356723 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:47.356771 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:47.360630 (+ 3859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.361584 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.361588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.361591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.361722 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.362274 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.362280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.362284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.362683 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:47.362980 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.363342 (+ 362us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.363557 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:58:47.380214 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.337322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42652 (local address 127.8.208.124:45209)
0929 02:58:47.337515 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.337522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.364250 (+ 26728us) server_negotiation.cc:407] Connection header received
0929 02:58:47.364538 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.364544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.364631 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.364718 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:47.366658 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.367759 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.369554 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.370245 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.370371 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.373598 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:47.373605 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:47.373610 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:47.373646 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:47.377035 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.378139 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.378145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.378149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.378290 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.378701 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.378706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.378710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.379029 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:47.379283 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.379623 (+ 340us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.379843 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:58:47.479337 15156 catalog_manager.cc:4841] TS bfa9940c93b24f89b616b77e3b7d5e98 (127.8.208.65:39239): tablet 6110845402a84bb191bd31eaa977a6a8 (table test-table [id=5517df5aeb6b4c95a45155322f1a43eb]) successfully deleted
I20260929 02:58:47.481767 15158 catalog_manager.cc:4841] TS fac6550165ad4ba98559f94c8870bc46 (127.8.208.66:46003): tablet 6110845402a84bb191bd31eaa977a6a8 (table test-table [id=5517df5aeb6b4c95a45155322f1a43eb]) successfully deleted
I20260929 02:58:47.482496 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 15139
I20260929 02:58:47.521407 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.505835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42654 (local address 127.8.208.124:45209)
0929 02:58:47.506047 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.506053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.506116 (+ 63us) server_negotiation.cc:407] Connection header received
0929 02:58:47.506309 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.506315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.506407 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.506511 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:47.508614 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.509566 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.510975 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.511402 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.511540 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.515346 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:47.515354 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:47.515359 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:47.515398 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:47.518549 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.519194 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.519201 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.519209 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.519324 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.519687 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.519693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.519696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.520072 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:47.520316 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.520760 (+ 444us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.521017 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:58:47.522891 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:47.504955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.8.208.125:38021)
0929 02:58:47.505188 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:47.505195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:47.505490 (+ 295us) server_negotiation.cc:407] Connection header received
0929 02:58:47.505777 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:47.505783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:47.505878 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:47.505974 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:47.508003 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.508930 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.510921 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:47.511617 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:47.511713 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:47.516161 (+ 4448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:47.516173 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:47.516178 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:47.516249 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:47.519654 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.520358 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.520363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.520366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.520528 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:47.520989 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:47.520995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:47.521004 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:47.521366 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:47.521594 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:47.522321 (+ 727us) server_negotiation.cc:299] Negotiation successful
0929 02:58:47.522532 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:58:48.296952 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.296127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.67:42265)
0929 02:58:48.296605 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.296721 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":238,"threads_started":1}
W20260929 02:58:48.297395 16428 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:33867 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
I20260929 02:58:48.298717 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.297976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.66:44597)
0929 02:58:48.298394 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.298536 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":208,"threads_started":1}
W20260929 02:58:48.299316 16287 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:33867 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
I20260929 02:58:48.302948 16499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.302182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.65:37519)
0929 02:58:48.302584 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.302738 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":198,"threads_started":1}
W20260929 02:58:48.303341 16141 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:33867 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
I20260929 02:58:48.814796 16500 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 802fa531790e45aab9bdccc54d19ae50)
I20260929 02:58:48.815091 16500 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } }
I20260929 02:58:48.815968 16501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 802fa531790e45aab9bdccc54d19ae50)
I20260929 02:58:48.816187 16501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } }
I20260929 02:58:48.816524 16500 leader_election.cc:288] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 802fa531790e45aab9bdccc54d19ae50 (127.8.208.126:33867), 2dc0f606d3ab453c8935c02a5f5671a8 (127.8.208.124:45209)
I20260929 02:58:48.818252 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.816763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:48986)
0929 02:58:48.817664 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.817908 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":585,"threads_started":1}
I20260929 02:58:48.819226 16501 leader_election.cc:288] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 802fa531790e45aab9bdccc54d19ae50 (127.8.208.126:33867), f5bff55efd8448438781b118662b2742 (127.8.208.125:38021)
I20260929 02:58:48.820397 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.817537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:48992)
0929 02:58:48.819885 (+ 2348us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.820063 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2222,"thread_start_us":2089,"threads_started":1}
W20260929 02:58:48.824569 15947 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
W20260929 02:58:48.832396 15753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
I20260929 02:58:48.836467 16504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.835713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:49004)
0929 02:58:48.835895 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.836154 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 02:58:48.836753 15753 leader_election.cc:334] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 802fa531790e45aab9bdccc54d19ae50 (127.8.208.126:33867): Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
I20260929 02:58:48.837044 16506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.836355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:49014)
0929 02:58:48.836558 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.836762 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 02:58:48.837343 15947 leader_election.cc:334] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 802fa531790e45aab9bdccc54d19ae50 (127.8.208.126:33867): Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
I20260929 02:58:48.838167 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.817776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:45209 (local address 127.0.0.1:42666)
0929 02:58:48.818640 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.818664 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:48.818821 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:48.830896 (+ 12075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:48.830901 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:48.830941 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:48.831189 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:48.831201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.833042 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.833050 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:48.835568 (+ 2518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:48.835585 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.837563 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.837572 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:48.837756 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:48.837787 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:58:48.837877 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 02:58:48.837986 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":14328,"threads_started":1}
I20260929 02:58:48.838390 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.818030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42666 (local address 127.8.208.124:45209)
0929 02:58:48.830256 (+ 12226us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:48.830262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:48.830282 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:58:48.830342 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:48.830347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:48.830416 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:48.831338 (+ 922us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:48.831590 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.832816 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.835774 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.837368 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.837505 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:48.837617 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:48.837937 (+ 320us) server_negotiation.cc:299] Negotiation successful
0929 02:58:48.838073 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12104,"thread_start_us":12705,"threads_started":1}
I20260929 02:58:48.839046 15972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5bff55efd8448438781b118662b2742" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" is_pre_election: true
I20260929 02:58:48.839520 15972 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5bff55efd8448438781b118662b2742 in term 1.
I20260929 02:58:48.839998 15753 leader_election.cc:302] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2dc0f606d3ab453c8935c02a5f5671a8, f5bff55efd8448438781b118662b2742; no voters: 802fa531790e45aab9bdccc54d19ae50
I20260929 02:58:48.840363 16500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 02:58:48.840487 16500 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 802fa531790e45aab9bdccc54d19ae50)
I20260929 02:58:48.840590 16500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:58:48.842821 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.820018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:38021 (local address 127.0.0.1:53366)
0929 02:58:48.820749 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.820770 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:48.820956 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:48.836495 (+ 15539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:48.836500 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:48.836533 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:48.836982 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:48.836995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.838744 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.838748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:48.840595 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:48.840606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.842323 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.842327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:48.842448 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:48.842471 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:58:48.842543 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 02:58:48.842633 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":4229,"threads_started":1}
I20260929 02:58:48.843065 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.821242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53366 (local address 127.8.208.125:38021)
0929 02:58:48.836157 (+ 14915us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:48.836163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:48.836180 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:58:48.836238 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:48.836243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:48.836312 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:48.837148 (+ 836us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:48.837433 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.838617 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.840752 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.842109 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.842270 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:48.842382 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:48.842587 (+ 205us) server_negotiation.cc:299] Negotiation successful
0929 02:58:48.842716 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14755,"thread_start_us":13812,"threads_started":1}
I20260929 02:58:48.843636 15778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f5bff55efd8448438781b118662b2742" is_pre_election: true
I20260929 02:58:48.843888 16500 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } }
I20260929 02:58:48.844332 15778 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2dc0f606d3ab453c8935c02a5f5671a8 in current term 2: Already voted for candidate f5bff55efd8448438781b118662b2742 in this term.
I20260929 02:58:48.844626 16500 leader_election.cc:288] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [CANDIDATE]: Term 2 election: Requested vote from peers 802fa531790e45aab9bdccc54d19ae50 (127.8.208.126:33867), 2dc0f606d3ab453c8935c02a5f5671a8 (127.8.208.124:45209)
I20260929 02:58:48.844956 15947 leader_election.cc:302] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2dc0f606d3ab453c8935c02a5f5671a8; no voters: 802fa531790e45aab9bdccc54d19ae50, f5bff55efd8448438781b118662b2742
I20260929 02:58:48.845155 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.844750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:49020)
0929 02:58:48.844898 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.844985 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 02:58:48.845350 16501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:58:48.845270 15972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5bff55efd8448438781b118662b2742" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2dc0f606d3ab453c8935c02a5f5671a8"
I20260929 02:58:48.846099 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.845771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:49022)
0929 02:58:48.845906 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.845987 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 02:58:48.846222 15753 leader_election.cc:334] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 802fa531790e45aab9bdccc54d19ae50 (127.8.208.126:33867): Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
I20260929 02:58:48.847688 16501 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 02:58:48.849833 15972 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5bff55efd8448438781b118662b2742 in term 2.
I20260929 02:58:48.850528 15753 leader_election.cc:302] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2dc0f606d3ab453c8935c02a5f5671a8, f5bff55efd8448438781b118662b2742; no voters: 802fa531790e45aab9bdccc54d19ae50
I20260929 02:58:48.850837 16500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 02:58:48.851256 16500 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [term 2 LEADER]: Becoming Leader. State: Replica: f5bff55efd8448438781b118662b2742, State: Running, Role: LEADER
I20260929 02:58:48.851626 16500 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } }
I20260929 02:58:48.853477 16510 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5bff55efd8448438781b118662b2742. Latest consensus state: current_term: 2 leader_uuid: "f5bff55efd8448438781b118662b2742" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } } }
I20260929 02:58:48.853878 16510 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:48.854526 16512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:48.859045 16512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:48.859843 16512 catalog_manager.cc:1259] Loaded cluster ID: b7690fbfec114c13b738f2f439821c9c
I20260929 02:58:48.859942 16512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:48.861124 16512 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:48.862071 16512 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: Loaded TSK: 0
I20260929 02:58:48.863054 16512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:58:48.918249 15767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:58:48.925429 15767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:48.929529 15972 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [term 2 FOLLOWER]: Refusing update from remote peer f5bff55efd8448438781b118662b2742: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260929 02:58:48.929622 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.929116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:49048)
0929 02:58:48.929331 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.929443 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260929 02:58:48.930332 16510 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260929 02:58:48.930805 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.930438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:49052)
0929 02:58:48.930593 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.930663 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 02:58:48.931052 15753 consensus_peers.cc:487] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 -> Peer 802fa531790e45aab9bdccc54d19ae50 (127.8.208.126:33867): Couldn't send request to peer 802fa531790e45aab9bdccc54d19ae50. Status: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 02:58:48.932897 16501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5bff55efd8448438781b118662b2742. Latest consensus state: current_term: 2 leader_uuid: "f5bff55efd8448438781b118662b2742" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } } }
I20260929 02:58:48.933210 16501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:58:48.934710 16510 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "f5bff55efd8448438781b118662b2742" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } } }
I20260929 02:58:48.934989 16510 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:48.935340 16501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f5bff55efd8448438781b118662b2742" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "802fa531790e45aab9bdccc54d19ae50" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 33867 } } peers { permanent_uuid: "f5bff55efd8448438781b118662b2742" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "2dc0f606d3ab453c8935c02a5f5671a8" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 45209 } attrs { promote: false } } }
I20260929 02:58:48.935623 16501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc0f606d3ab453c8935c02a5f5671a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:58:48.936278 16514 mvcc.cc:205] Tried to move back new op lower bound from 7334505385688662016 to 7334505385379438592. Current Snapshot: MvccSnapshot[applied={T|T < 7334505385688662016 or (T in {7334505385688662016})}]
I20260929 02:58:48.950032 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.943580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:46003 (local address 127.0.0.1:51954)
0929 02:58:48.943955 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.943972 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:48.944087 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:48.944893 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:48.944898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:48.944925 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:48.945125 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:48.945135 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.946556 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.946562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:48.947974 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:48.947985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.949364 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.949369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:48.949574 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:48.949615 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 02:58:48.949692 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 02:58:48.949801 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":1}
I20260929 02:58:48.951108 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.943915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51954 (local address 127.8.208.66:46003)
0929 02:58:48.944507 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:48.944511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:48.944530 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:58:48.944605 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:48.944611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:48.944704 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:48.945294 (+ 590us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:48.945464 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.946421 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.948142 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.949155 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.950620 (+ 1465us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:48.950767 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:48.950805 (+ 38us) server_negotiation.cc:299] Negotiation successful
0929 02:58:48.950902 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":391,"threads_started":1}
I20260929 02:58:48.951790 16221 tablet_service.cc:1467] Processing CreateTablet for tablet 0aad31888a9e46b185b4e6dd1cac933b (DEFAULT_TABLE table=test-table [id=f58744d8a9f54cacae670d2b0f808c8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:48.952255 16221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aad31888a9e46b185b4e6dd1cac933b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:48.953218 16504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.943127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:39239 (local address 127.0.0.1:42578)
0929 02:58:48.943720 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.943743 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:48.943873 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:48.944354 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:48.944358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:48.944384 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:48.944757 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:48.944769 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.947040 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.947076 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:48.949950 (+ 2874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:48.949970 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.952177 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.952186 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:48.952442 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:48.952496 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 02:58:48.952572 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 02:58:48.952762 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487}
I20260929 02:58:48.954463 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.943245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42578 (local address 127.8.208.65:39239)
0929 02:58:48.943680 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:48.943687 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:48.943997 (+ 310us) server_negotiation.cc:407] Connection header received
0929 02:58:48.944098 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:48.944105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:48.944199 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:48.944926 (+ 727us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:48.945193 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.946829 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.950211 (+ 3382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.951879 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.953617 (+ 1738us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:48.953774 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:48.953837 (+ 63us) server_negotiation.cc:299] Negotiation successful
0929 02:58:48.954016 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":239,"threads_started":1}
I20260929 02:58:48.955263 16075 tablet_service.cc:1467] Processing CreateTablet for tablet 0aad31888a9e46b185b4e6dd1cac933b (DEFAULT_TABLE table=test-table [id=f58744d8a9f54cacae670d2b0f808c8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:48.956022 16075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aad31888a9e46b185b4e6dd1cac933b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:48.957881 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.943979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:46209 (local address 127.0.0.1:56914)
0929 02:58:48.950272 (+ 6293us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.950289 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:48.950393 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:48.951801 (+ 1408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:48.951808 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:48.951847 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:48.952020 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:48.952029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.953607 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.953614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:48.955433 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:48.955447 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.957129 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.957137 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:48.957375 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:48.957409 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:58:48.957476 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 02:58:48.957599 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6197,"thread_start_us":12935,"threads_started":1}
I20260929 02:58:48.958840 16522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.944412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56914 (local address 127.8.208.67:46209)
0929 02:58:48.944912 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:48.944918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:48.951487 (+ 6569us) server_negotiation.cc:407] Connection header received
0929 02:58:48.951566 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:48.951571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:48.951655 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:48.952163 (+ 508us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:48.952350 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.953370 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.955623 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:48.956920 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:48.958249 (+ 1329us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:48.958355 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:48.958404 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 02:58:48.958532 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":279,"threads_started":1}
I20260929 02:58:48.959741 16362 tablet_service.cc:1467] Processing CreateTablet for tablet 0aad31888a9e46b185b4e6dd1cac933b (DEFAULT_TABLE table=test-table [id=f58744d8a9f54cacae670d2b0f808c8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:48.960194 16362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aad31888a9e46b185b4e6dd1cac933b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:48.960227 16523 tablet_bootstrap.cc:492] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46: Bootstrap starting.
I20260929 02:58:48.962431 16523 tablet_bootstrap.cc:654] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:48.965461 16523 tablet_bootstrap.cc:492] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46: No bootstrap required, opened a new log
I20260929 02:58:48.965763 16523 ts_tablet_manager.cc:1397] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 02:58:48.966888 16523 raft_consensus.cc:348] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } }
I20260929 02:58:48.967232 16523 raft_consensus.cc:374] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:48.967427 16524 tablet_bootstrap.cc:492] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98: Bootstrap starting.
I20260929 02:58:48.967408 16523 raft_consensus.cc:729] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fac6550165ad4ba98559f94c8870bc46, State: Initialized, Role: FOLLOWER
I20260929 02:58:48.967662 16523 consensus_queue.cc:260] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } }
I20260929 02:58:48.969209 16526 tablet_bootstrap.cc:492] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d: Bootstrap starting.
I20260929 02:58:48.970467 16524 tablet_bootstrap.cc:654] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:48.970853 16526 tablet_bootstrap.cc:654] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:48.972980 16526 tablet_bootstrap.cc:492] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d: No bootstrap required, opened a new log
I20260929 02:58:48.973117 16526 ts_tablet_manager.cc:1397] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260929 02:58:48.973435 16523 ts_tablet_manager.cc:1428] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260929 02:58:48.973949 16526 raft_consensus.cc:348] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } }
I20260929 02:58:48.974110 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.972878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.66:32837)
0929 02:58:48.973699 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.973843 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":542,"threads_started":1}
I20260929 02:58:48.974153 16526 raft_consensus.cc:374] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:48.974270 16526 raft_consensus.cc:729] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521bef1b580742b8b758d014adc3d92d, State: Initialized, Role: FOLLOWER
I20260929 02:58:48.974485 16526 consensus_queue.cc:260] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } }
I20260929 02:58:48.975958 16524 tablet_bootstrap.cc:492] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98: No bootstrap required, opened a new log
I20260929 02:58:48.976140 16524 ts_tablet_manager.cc:1397] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260929 02:58:48.976596 16288 heartbeater.cc:503] Master 127.8.208.125:38021 was elected leader, sending a full tablet report...
I20260929 02:58:48.976997 16524 raft_consensus.cc:348] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } }
I20260929 02:58:48.977349 16524 raft_consensus.cc:374] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:48.977510 16524 raft_consensus.cc:729] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa9940c93b24f89b616b77e3b7d5e98, State: Initialized, Role: FOLLOWER
I20260929 02:58:48.977849 16524 consensus_queue.cc:260] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } }
I20260929 02:58:48.978616 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.975355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.66:50379)
0929 02:58:48.978298 (+ 2943us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.978409 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2782}
I20260929 02:58:48.981711 16524 ts_tablet_manager.cc:1428] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260929 02:58:48.982395 16526 ts_tablet_manager.cc:1428] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d: Time spent starting tablet: real 0.009s user 0.003s sys 0.006s
I20260929 02:58:48.983932 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.982942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.67:55747)
0929 02:58:48.983541 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.983666 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":294,"threads_started":1}
I20260929 02:58:48.985229 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.982022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.65:33977)
0929 02:58:48.983285 (+ 1263us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.984919 (+ 1634us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1136,"thread_start_us":1011,"threads_started":1}
I20260929 02:58:48.985963 16429 heartbeater.cc:503] Master 127.8.208.125:38021 was elected leader, sending a full tablet report...
I20260929 02:58:48.986121 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.985340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.67:51859)
0929 02:58:48.985854 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.985940 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424}
I20260929 02:58:48.987388 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:48.986746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.65:51821)
0929 02:58:48.987215 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:48.987278 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386}
W20260929 02:58:49.085032 16143 tablet.cc:2369] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:58:49.192628 16532 raft_consensus.cc:482] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:49.192919 16532 raft_consensus.cc:504] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } }
I20260929 02:58:49.194531 16532 leader_election.cc:288] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fac6550165ad4ba98559f94c8870bc46 (127.8.208.66:46003), 521bef1b580742b8b758d014adc3d92d (127.8.208.67:46209)
I20260929 02:58:49.202333 16522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.195050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:52759 (local address 127.8.208.67:46209)
0929 02:58:49.195243 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:49.195249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:49.195768 (+ 519us) server_negotiation.cc:407] Connection header received
0929 02:58:49.195937 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:49.195944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:49.196025 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:49.196098 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:49.196685 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.197697 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.199952 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.201243 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.201407 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:49.201472 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:49.201954 (+ 482us) server_negotiation.cc:299] Negotiation successful
0929 02:58:49.202084 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:58:49.202344 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.194900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:36469 (local address 127.8.208.66:46003)
0929 02:58:49.195125 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:49.195133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:49.195401 (+ 268us) server_negotiation.cc:407] Connection header received
0929 02:58:49.195655 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:49.195663 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:49.195742 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:49.195836 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:49.196381 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.197443 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.199967 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.201243 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.201400 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:49.201472 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:49.201954 (+ 482us) server_negotiation.cc:299] Negotiation successful
0929 02:58:49.202084 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:58:49.202454 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.194967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:46003 (local address 127.8.208.65:36469)
0929 02:58:49.195285 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:49.195307 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:49.195499 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:49.195879 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:49.195883 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:49.195911 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:49.196227 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:49.196241 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.197555 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.197559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:49.199784 (+ 2225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:49.199801 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.201542 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.201548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:49.201765 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:49.201811 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 02:58:49.201913 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 02:58:49.202105 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":31}
I20260929 02:58:49.202498 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.195016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:46209 (local address 127.8.208.65:52759)
0929 02:58:49.195652 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:49.195675 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:49.195782 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:49.196161 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:49.196168 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:49.196203 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:49.196543 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:49.196551 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.197820 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.197825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:49.199820 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:49.199829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.201510 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.201523 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:49.201750 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:49.201811 (+ 61us) client_negotiation.cc:769] Sending connection context
0929 02:58:49.201913 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 02:58:49.202105 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":74,"thread_start_us":358,"threads_started":1}
I20260929 02:58:49.203173 16382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aad31888a9e46b185b4e6dd1cac933b" candidate_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "521bef1b580742b8b758d014adc3d92d" is_pre_election: true
I20260929 02:58:49.203420 16382 raft_consensus.cc:2454] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa9940c93b24f89b616b77e3b7d5e98 in term 0.
I20260929 02:58:49.203272 16241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aad31888a9e46b185b4e6dd1cac933b" candidate_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fac6550165ad4ba98559f94c8870bc46" is_pre_election: true
I20260929 02:58:49.203693 16241 raft_consensus.cc:2454] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa9940c93b24f89b616b77e3b7d5e98 in term 0.
I20260929 02:58:49.203962 16030 leader_election.cc:302] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 521bef1b580742b8b758d014adc3d92d, bfa9940c93b24f89b616b77e3b7d5e98; no voters:
I20260929 02:58:49.204378 16532 raft_consensus.cc:2781] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:49.204519 16532 raft_consensus.cc:482] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:49.204629 16532 raft_consensus.cc:3037] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:49.206713 16532 raft_consensus.cc:504] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } }
I20260929 02:58:49.207278 16532 leader_election.cc:288] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [CANDIDATE]: Term 1 election: Requested vote from peers fac6550165ad4ba98559f94c8870bc46 (127.8.208.66:46003), 521bef1b580742b8b758d014adc3d92d (127.8.208.67:46209)
I20260929 02:58:49.207716 16241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aad31888a9e46b185b4e6dd1cac933b" candidate_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fac6550165ad4ba98559f94c8870bc46"
I20260929 02:58:49.207731 16382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aad31888a9e46b185b4e6dd1cac933b" candidate_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "521bef1b580742b8b758d014adc3d92d"
I20260929 02:58:49.207890 16241 raft_consensus.cc:3037] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:49.207913 16382 raft_consensus.cc:3037] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:49.209837 16382 raft_consensus.cc:2454] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa9940c93b24f89b616b77e3b7d5e98 in term 1.
I20260929 02:58:49.210031 16241 raft_consensus.cc:2454] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa9940c93b24f89b616b77e3b7d5e98 in term 1.
I20260929 02:58:49.210247 16030 leader_election.cc:302] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 521bef1b580742b8b758d014adc3d92d, bfa9940c93b24f89b616b77e3b7d5e98; no voters:
I20260929 02:58:49.210565 16532 raft_consensus.cc:2781] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:49.210973 16532 raft_consensus.cc:686] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 1 LEADER]: Becoming Leader. State: Replica: bfa9940c93b24f89b616b77e3b7d5e98, State: Running, Role: LEADER
I20260929 02:58:49.211292 16142 heartbeater.cc:503] Master 127.8.208.125:38021 was elected leader, sending a full tablet report...
I20260929 02:58:49.211426 16532 consensus_queue.cc:237] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } }
I20260929 02:58:49.212019 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.211643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.65:60291)
0929 02:58:49.211802 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:49.211876 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 02:58:49.214927 15768 catalog_manager.cc:5495] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa9940c93b24f89b616b77e3b7d5e98 (127.8.208.65). New cstate: current_term: 1 leader_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9940c93b24f89b616b77e3b7d5e98" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 39239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 } health_report { overall_health: UNKNOWN } } }
I20260929 02:58:49.250301 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.245404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.8.208.65:39239)
0929 02:58:49.245625 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:49.245630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:49.245647 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:58:49.245842 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:49.245846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:49.245935 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:49.246055 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:49.246359 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.247180 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.248391 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.248786 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.248876 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:49.249222 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:49.249724 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:49.249828 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:49.249903 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 02:58:49.249980 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:58:49.254330 16382 raft_consensus.cc:1261] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d [term 1 FOLLOWER]: Refusing update from remote peer bfa9940c93b24f89b616b77e3b7d5e98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:49.254621 16241 raft_consensus.cc:1261] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46 [term 1 FOLLOWER]: Refusing update from remote peer bfa9940c93b24f89b616b77e3b7d5e98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:49.255095 16536 consensus_queue.cc:1035] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "521bef1b580742b8b758d014adc3d92d" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 46209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:49.255448 16532 consensus_queue.cc:1035] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fac6550165ad4ba98559f94c8870bc46" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:49.258046 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.257584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.67:49507)
0929 02:58:49.257749 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:49.257851 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 02:58:49.258601 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.258102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.66:59467)
0929 02:58:49.258321 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:49.258434 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260929 02:58:49.265563 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.264804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.65:34253)
0929 02:58:49.265194 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:49.265319 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208}
I20260929 02:58:49.265988 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.265074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.66:55091)
0929 02:58:49.265743 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:49.265831 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567}
I20260929 02:58:49.267212 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.266850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.67:54339)
0929 02:58:49.266981 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:49.267068 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 02:58:49.270710 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.270277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.8.208.65:49055)
0929 02:58:49.270487 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:49.270571 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260929 02:58:49.270897 16540 mvcc.cc:205] Tried to move back new op lower bound from 7334505387018063872 to 7334505386854055936. Current Snapshot: MvccSnapshot[applied={T|T < 7334505387018063872 or (T in {7334505387018063872})}]
I20260929 02:58:49.278761 15768 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:49.279019 15768 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:49.291162 15768 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5bff55efd8448438781b118662b2742: Sending DeleteTablet for 3 replicas of tablet 0aad31888a9e46b185b4e6dd1cac933b
I20260929 02:58:49.292209 16075 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aad31888a9e46b185b4e6dd1cac933b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:49 UTC) from {username='oryx', principal='oryx/127.8.208.125@KRBTEST.COM'} at 127.0.0.1:42578
I20260929 02:58:49.292690 16221 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aad31888a9e46b185b4e6dd1cac933b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:49 UTC) from {username='oryx', principal='oryx/127.8.208.125@KRBTEST.COM'} at 127.0.0.1:51954
I20260929 02:58:49.292935 16362 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aad31888a9e46b185b4e6dd1cac933b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:49 UTC) from {username='oryx', principal='oryx/127.8.208.125@KRBTEST.COM'} at 127.0.0.1:56914
I20260929 02:58:49.295001 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 15934
I20260929 02:58:49.302726 16549 tablet_replica.cc:330] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98: stopping tablet replica
I20260929 02:58:49.303138 16549 raft_consensus.cc:2229] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:49.303829 16550 tablet_replica.cc:330] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46: stopping tablet replica
I20260929 02:58:49.303932 16549 raft_consensus.cc:2258] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:49.304162 16550 raft_consensus.cc:2229] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:49.304769 16550 raft_consensus.cc:2258] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:49.305069 16551 tablet_replica.cc:330] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d: stopping tablet replica
I20260929 02:58:49.305385 16551 raft_consensus.cc:2229] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:49.305548 16549 ts_tablet_manager.cc:1905] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:49.305758 16551 raft_consensus.cc:2258] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:49.306413 16550 ts_tablet_manager.cc:1905] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:49.306938 16551 ts_tablet_manager.cc:1905] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:49.312909 16549 ts_tablet_manager.cc:1918] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:49.313118 16549 log.cc:1196] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/0aad31888a9e46b185b4e6dd1cac933b
I20260929 02:58:49.313445 16551 ts_tablet_manager.cc:1918] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:49.313664 16551 log.cc:1196] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/0aad31888a9e46b185b4e6dd1cac933b
I20260929 02:58:49.313743 16549 ts_tablet_manager.cc:1939] T 0aad31888a9e46b185b4e6dd1cac933b P bfa9940c93b24f89b616b77e3b7d5e98: Deleting consensus metadata
I20260929 02:58:49.314142 16551 ts_tablet_manager.cc:1939] T 0aad31888a9e46b185b4e6dd1cac933b P 521bef1b580742b8b758d014adc3d92d: Deleting consensus metadata
I20260929 02:58:49.315060 15753 catalog_manager.cc:4841] TS bfa9940c93b24f89b616b77e3b7d5e98 (127.8.208.65:39239): tablet 0aad31888a9e46b185b4e6dd1cac933b (table test-table [id=f58744d8a9f54cacae670d2b0f808c8d]) successfully deleted
I20260929 02:58:49.315553 15754 catalog_manager.cc:4841] TS 521bef1b580742b8b758d014adc3d92d (127.8.208.67:46209): tablet 0aad31888a9e46b185b4e6dd1cac933b (table test-table [id=f58744d8a9f54cacae670d2b0f808c8d]) successfully deleted
I20260929 02:58:49.315796 16550 ts_tablet_manager.cc:1918] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:49.315950 16550 log.cc:1196] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/0aad31888a9e46b185b4e6dd1cac933b
I20260929 02:58:49.316421 16550 ts_tablet_manager.cc:1939] T 0aad31888a9e46b185b4e6dd1cac933b P fac6550165ad4ba98559f94c8870bc46: Deleting consensus metadata
I20260929 02:58:49.317518 15755 catalog_manager.cc:4841] TS fac6550165ad4ba98559f94c8870bc46 (127.8.208.66:46003): tablet 0aad31888a9e46b185b4e6dd1cac933b (table test-table [id=f58744d8a9f54cacae670d2b0f808c8d]) successfully deleted
I20260929 02:58:49.328197 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.296779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.8.208.65:39239)
0929 02:58:49.297017 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:49.297026 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:49.305641 (+ 8615us) server_negotiation.cc:407] Connection header received
0929 02:58:49.305831 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:49.305835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:49.305925 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:49.306042 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:49.312742 (+ 6700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.313918 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.318502 (+ 4584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.319095 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.319222 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:49.322204 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:49.322216 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:49.322235 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:49.322269 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:49.325361 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:49.326083 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:49.326092 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:49.326097 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:49.326249 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:49.326639 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:49.326644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:49.326647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:49.326998 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:49.327320 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:49.327650 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 02:58:49.327861 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 02:58:49.328936 16135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:49.428875 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.428478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:49060)
0929 02:58:49.428643 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:49.428729 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 02:58:49.430035 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.429697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:33867 (local address 127.0.0.1:49072)
0929 02:58:49.429843 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:49.429928 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 02:58:49.438995 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 15934
I20260929 02:58:49.457504 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 16152
I20260929 02:58:49.470778 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.458264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51958 (local address 127.8.208.66:46003)
0929 02:58:49.458458 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:49.458464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:49.458586 (+ 122us) server_negotiation.cc:407] Connection header received
0929 02:58:49.458793 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:49.458803 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:49.458863 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:49.458980 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:49.460310 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.461045 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.462673 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.463063 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.463161 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:49.465547 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:49.465559 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:49.465575 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:49.465609 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:49.468457 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:49.469035 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:49.469042 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:49.469044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:49.469153 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:49.469511 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:49.469514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:49.469516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:49.469847 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:49.470087 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:49.470245 (+ 158us) server_negotiation.cc:299] Negotiation successful
0929 02:58:49.470405 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:58:49.471400 16281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:49.590356 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 16152
I20260929 02:58:49.612114 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 16299
I20260929 02:58:49.625605 16522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:49.612859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56918 (local address 127.8.208.67:46209)
0929 02:58:49.613053 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:49.613058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:49.613159 (+ 101us) server_negotiation.cc:407] Connection header received
0929 02:58:49.613425 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:49.613432 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:49.613511 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:49.613622 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:49.615004 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.615695 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.616994 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:49.617452 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:49.617587 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:49.619990 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:49.620004 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:49.620020 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:49.620051 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:49.622669 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:49.623325 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:49.623340 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:49.623343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:49.623474 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:49.623857 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:49.623863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:49.623866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:49.624255 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:49.624568 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:49.624879 (+ 311us) server_negotiation.cc:299] Negotiation successful
0929 02:58:49.625115 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 02:58:49.626241 16422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:49.720423 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 16299
I20260929 02:58:49.739734 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 15632
I20260929 02:58:49.740890 15798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:49.863756 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 15632
I20260929 02:58:49.887089 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 15732
I20260929 02:58:49.888207 15992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:50.002892 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 15732
2026-09-29T02:58:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9426 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:58:50 dist-test-slave-m1rv krb5kdc[16560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:58:50 dist-test-slave-m1rv krb5kdc[16560](info): set up 2 sockets
Sep 29 02:58:50 dist-test-slave-m1rv krb5kdc[16560](info): commencing operation
krb5kdc: starting...
W20260929 02:58:51.110167 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:58:51 dist-test-slave-m1rv krb5kdc[16560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:58:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.126.keytab.
Entry for principal oryx/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.126.keytab.
I20260929 02:58:51.318905 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:39247
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.126.keytab
--principal=oryx/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:39859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:39247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:51.654017 16573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:51.654282 16573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:51.654477 16573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:51.661937 16573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:51.662056 16573 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 02:58:51.662124 16573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:51.662259 16573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:51.662345 16573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:51.662437 16573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:51.672820 16573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:39859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:39247
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.126.keytab
--principal=oryx/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:39247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:51.674033 16573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:51.675223 16573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:51.686789 16578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:51.686892 16581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:51.686975 16579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:51.687786 16573 server_base.cc:1034] running on GCE node
I20260929 02:58:51.688427 16573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:51.689689 16573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:51.690899 16573 hybrid_clock.cc:648] HybridClock initialized: now 1790650731690841 us; error 25 us; skew 500 ppm
I20260929 02:58:51.691164 16573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:51 dist-test-slave-m1rv krb5kdc[16560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650731, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:51.694252 16573 init.cc:377] Logged in from keytab as oryx/127.8.208.126@KRBTEST.COM (short username oryx)
I20260929 02:58:51.695660 16573 webserver.cc:460] Webserver started at http://127.8.208.126:42689/ using document root <none> and password file <none>
I20260929 02:58:51.696324 16573 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:51.696472 16573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:51.696792 16573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:51.699707 16573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "a1327141dd1f4ed399c610d0cd1a3822"
format_stamp: "Formatted at 2026-09-29 02:58:51 on dist-test-slave-m1rv"
server_key: "7c9a0a9938e4732fde60363553f7308c"
server_key_iv: "7b5d94e51fb1e8054ec1b24dd9737515"
server_key_version: "encryptionkey@0"
I20260929 02:58:51.700482 16573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "a1327141dd1f4ed399c610d0cd1a3822"
format_stamp: "Formatted at 2026-09-29 02:58:51 on dist-test-slave-m1rv"
server_key: "7c9a0a9938e4732fde60363553f7308c"
server_key_iv: "7b5d94e51fb1e8054ec1b24dd9737515"
server_key_version: "encryptionkey@0"
I20260929 02:58:51.705381 16573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 02:58:51.708544 16588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:51.709782 16573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:58:51.709960 16573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "a1327141dd1f4ed399c610d0cd1a3822"
format_stamp: "Formatted at 2026-09-29 02:58:51 on dist-test-slave-m1rv"
server_key: "7c9a0a9938e4732fde60363553f7308c"
server_key_iv: "7b5d94e51fb1e8054ec1b24dd9737515"
server_key_version: "encryptionkey@0"
I20260929 02:58:51.710167 16573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:51.723320 16573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:51.727365 16573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:51.727635 16573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:51.748924 16573 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:39247
I20260929 02:58:51.748952 16649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:39247 every 8 connection(s)
I20260929 02:58:51.750257 16573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:58:51.754024 16650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:51.758797 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 16573
I20260929 02:58:51.759019 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:58:51.759702 9025 external_mini_cluster.cc:1428] Setting key 56b020b312ce5905f44a1c1f79dd1aa6
I20260929 02:58:51.765831 16650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822: Bootstrap starting.
I20260929 02:58:51.769515 16650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:51.770546 16650 log.cc:824] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:51.773466 16650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822: No bootstrap required, opened a new log
Sep 29 02:58:51 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
I20260929 02:58:51.782678 16650 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1327141dd1f4ed399c610d0cd1a3822" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39247 } }
I20260929 02:58:51.783041 16650 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:51.783160 16650 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1327141dd1f4ed399c610d0cd1a3822, State: Initialized, Role: FOLLOWER
I20260929 02:58:51.783357 16653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:51.761725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52770 (local address 127.8.208.126:39247)
0929 02:58:51.762593 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:51.762608 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:51.762660 (+ 52us) server_negotiation.cc:407] Connection header received
0929 02:58:51.765038 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:51.765064 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:51.765632 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:51.766046 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:51.767277 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:51.769312 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:51.770573 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:51.771551 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:51.771692 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:51.776025 (+ 4333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:51.776036 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:51.776049 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:51.776081 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:51.779219 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:51.780064 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:51.780070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:51.780076 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:51.780238 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:51.780644 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:51.780647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:51.780649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:51.781147 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:51.781435 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:51.781842 (+ 407us) server_negotiation.cc:299] Negotiation successful
0929 02:58:51.782238 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":343,"threads_started":1}
I20260929 02:58:51.783867 16650 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1327141dd1f4ed399c610d0cd1a3822" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39247 } }
I20260929 02:58:51.784169 16650 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:58:51.784307 16650 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:58:51.784475 16650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:51.787007 16650 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1327141dd1f4ed399c610d0cd1a3822" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39247 } }
I20260929 02:58:51.787432 16650 leader_election.cc:302] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1327141dd1f4ed399c610d0cd1a3822; no voters:
I20260929 02:58:51.787922 16650 leader_election.cc:288] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:58:51.788055 16655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:51.788944 16655 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [term 1 LEADER]: Becoming Leader. State: Replica: a1327141dd1f4ed399c610d0cd1a3822, State: Running, Role: LEADER
I20260929 02:58:51.789611 16655 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1327141dd1f4ed399c610d0cd1a3822" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39247 } }
I20260929 02:58:51.790084 16650 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:58:51.792295 16656 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1327141dd1f4ed399c610d0cd1a3822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1327141dd1f4ed399c610d0cd1a3822" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39247 } } }
I20260929 02:58:51.793458 16656 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:51.792325 16657 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1327141dd1f4ed399c610d0cd1a3822. Latest consensus state: current_term: 1 leader_uuid: "a1327141dd1f4ed399c610d0cd1a3822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1327141dd1f4ed399c610d0cd1a3822" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39247 } } }
I20260929 02:58:51.793931 16657 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822 [sys.catalog]: This master's current role is: LEADER
I20260929 02:58:51.797118 16663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:58:51.803864 16663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:58:51.812192 16663 catalog_manager.cc:1347] Generated new cluster ID: 55bfe7f5cdb548bfbbca5916dd9969c1
I20260929 02:58:51.812297 16663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:58:51.822726 16663 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:58:51.824052 16663 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:58:51.839960 16663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822: Generated new TSK 0
I20260929 02:58:51.840766 16663 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab.
Entry for principal oryx/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab.
I20260929 02:58:51.900291 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39247
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab
--principal=oryx/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:39859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:52.227871 16676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:52.228189 16676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:52.228374 16676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:52.235865 16676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:52.236042 16676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:52.236194 16676 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:58:52.236332 16676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:58:52.247006 16676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:39859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab
--principal=oryx/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.208.126:39247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:52.248317 16676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:52.249655 16676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:52.262890 16682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:52.262946 16681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:52.262969 16684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:52.263578 16676 server_base.cc:1034] running on GCE node
I20260929 02:58:52.264169 16676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:52.265106 16676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:52.266366 16676 hybrid_clock.cc:648] HybridClock initialized: now 1790650732266302 us; error 39 us; skew 500 ppm
I20260929 02:58:52.266656 16676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:52 dist-test-slave-m1rv krb5kdc[16560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650732, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:52.269853 16676 init.cc:377] Logged in from keytab as oryx/127.8.208.65@KRBTEST.COM (short username oryx)
I20260929 02:58:52.271219 16676 webserver.cc:460] Webserver started at http://127.8.208.65:46181/ using document root <none> and password file <none>
I20260929 02:58:52.271837 16676 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:52.271921 16676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:52.272277 16676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:52.275172 16676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "2716754effc24d2594c84fba1b15e571"
format_stamp: "Formatted at 2026-09-29 02:58:52 on dist-test-slave-m1rv"
server_key: "f6c14db774f0dd56c182715afe21a6fe"
server_key_iv: "943f7811b533124966dbcca0212ec466"
server_key_version: "encryptionkey@0"
I20260929 02:58:52.275954 16676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "2716754effc24d2594c84fba1b15e571"
format_stamp: "Formatted at 2026-09-29 02:58:52 on dist-test-slave-m1rv"
server_key: "f6c14db774f0dd56c182715afe21a6fe"
server_key_iv: "943f7811b533124966dbcca0212ec466"
server_key_version: "encryptionkey@0"
I20260929 02:58:52.280985 16676 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260929 02:58:52.284475 16691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:52.285811 16676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:58:52.286032 16676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "2716754effc24d2594c84fba1b15e571"
format_stamp: "Formatted at 2026-09-29 02:58:52 on dist-test-slave-m1rv"
server_key: "f6c14db774f0dd56c182715afe21a6fe"
server_key_iv: "943f7811b533124966dbcca0212ec466"
server_key_version: "encryptionkey@0"
I20260929 02:58:52.286247 16676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:52.304245 16676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:52.308251 16676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:52.308523 16676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:52.311285 16676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:52.311400 16676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:52.311555 16676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:52.311614 16676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:58:52 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650732, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
I20260929 02:58:52.336376 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:52.311626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:35283 (local address 127.8.208.126:39247)
0929 02:58:52.312083 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:52.312091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:52.313403 (+ 1312us) server_negotiation.cc:407] Connection header received
0929 02:58:52.315677 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:52.315682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:52.315771 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:52.315888 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:52.318312 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.319687 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.322003 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.322794 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.322948 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:52.327872 (+ 4924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:52.327882 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:52.327889 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:52.327941 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:52.332077 (+ 4136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:52.333087 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:52.333096 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:52.333102 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:52.333301 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:52.334015 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:52.334023 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:52.334028 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:52.334513 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:52.334677 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:52.335926 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0929 02:58:52.336126 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":266,"threads_started":1}
I20260929 02:58:52.338619 16701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:52.312348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39247 (local address 127.8.208.65:35283)
0929 02:58:52.313255 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:52.313292 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:52.315390 (+ 2098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:52.316187 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:52.316205 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:52.316993 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:52.318066 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:52.318113 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.319862 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.319868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:52.321861 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:52.321872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.323081 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.323092 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:52.323315 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:52.324215 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:52.324251 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:52.327575 (+ 3324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:52.332229 (+ 4654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:52.332243 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:52.332262 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:52.332924 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:52.333484 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:52.333492 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:52.333496 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:52.333842 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:52.334769 (+ 927us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:52.334785 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:52.335381 (+ 596us) client_negotiation.cc:769] Sending connection context
0929 02:58:52.335798 (+ 417us) client_negotiation.cc:240] Negotiation successful
0929 02:58:52.336551 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":320,"threads_started":1}
I20260929 02:58:52.363240 16778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:52.348842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39247 (local address 127.8.208.65:33571)
0929 02:58:52.349626 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:52.349652 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:52.349826 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:52.350150 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:52.350157 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:52.350896 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:52.351353 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:52.351369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.352633 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.352638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:52.354074 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:52.354089 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.354938 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.354945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:52.355123 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:52.355792 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:52.355807 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:52.356939 (+ 1132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:52.360201 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:52.360208 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:52.360212 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:52.360725 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:52.361232 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:52.361241 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:52.361245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:52.361458 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:52.362193 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:52.362196 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:52.362455 (+ 259us) client_negotiation.cc:769] Sending connection context
0929 02:58:52.362633 (+ 178us) client_negotiation.cc:240] Negotiation successful
0929 02:58:52.362906 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":376,"threads_started":1}
I20260929 02:58:52.364153 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:52.348815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:33571 (local address 127.8.208.126:39247)
0929 02:58:52.349030 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:52.349039 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:52.349747 (+ 708us) server_negotiation.cc:407] Connection header received
0929 02:58:52.349947 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:52.349951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:52.350006 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:52.350085 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:52.351553 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.352480 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.354222 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.354755 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.354904 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:52.357160 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:52.357164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:52.357205 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:52.357247 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:52.360078 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:52.360889 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:52.360899 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:52.360904 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:52.361076 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:52.361579 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:52.361584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:52.361588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:52.361986 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:52.362132 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:52.363554 (+ 1422us) server_negotiation.cc:299] Negotiation successful
0929 02:58:52.363814 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 02:58:52.365401 16676 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:32855
I20260929 02:58:52.365435 16811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:32855 every 8 connection(s)
I20260929 02:58:52.366673 16676 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:58:52.368741 16812 heartbeater.cc:347] Connected to a master server at 127.8.208.126:39247
I20260929 02:58:52.368960 16812 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:52.369635 16812 heartbeater.cc:511] Master 127.8.208.126:39247 requested a full tablet report, sending...
I20260929 02:58:52.371414 16605 ts_manager.cc:194] Registered new tserver with Master: 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855)
I20260929 02:58:52.372184 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 16676
I20260929 02:58:52.372400 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:58:52.373143 9025 external_mini_cluster.cc:1428] Setting key dceb679d5edaf77ceba85b70d40b8cd4
I20260929 02:58:52.373456 16605 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:35283
WARNING: no policy specified for oryx/127.8.208.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab.
Entry for principal oryx/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab.
I20260929 02:58:52.417902 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39247
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab
--principal=oryx/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:39859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:52.734035 16816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:52.734328 16816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:52.734504 16816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:52.741688 16816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:52.741827 16816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:52.741976 16816 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:58:52.742111 16816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:58:52.751518 16816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:39859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab
--principal=oryx/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.208.126:39247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:52.752679 16816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:52.754050 16816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:52.766427 16822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:52.766521 16821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:52.766724 16824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:52.768009 16816 server_base.cc:1034] running on GCE node
I20260929 02:58:52.768565 16816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:52.769479 16816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:52.770736 16816 hybrid_clock.cc:648] HybridClock initialized: now 1790650732770686 us; error 44 us; skew 500 ppm
I20260929 02:58:52.771054 16816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:52 dist-test-slave-m1rv krb5kdc[16560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650732, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:52.774109 16816 init.cc:377] Logged in from keytab as oryx/127.8.208.66@KRBTEST.COM (short username oryx)
I20260929 02:58:52.775485 16816 webserver.cc:460] Webserver started at http://127.8.208.66:33273/ using document root <none> and password file <none>
I20260929 02:58:52.776181 16816 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:52.776314 16816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:52.776639 16816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:52.779729 16816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "71484ace087340f396044b542cbe3a71"
format_stamp: "Formatted at 2026-09-29 02:58:52 on dist-test-slave-m1rv"
server_key: "1aae2a6c52e6ad765eaa72e2d04fc470"
server_key_iv: "ac6ad193afdab2702f67c5270dfa0d20"
server_key_version: "encryptionkey@0"
I20260929 02:58:52.780537 16816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "71484ace087340f396044b542cbe3a71"
format_stamp: "Formatted at 2026-09-29 02:58:52 on dist-test-slave-m1rv"
server_key: "1aae2a6c52e6ad765eaa72e2d04fc470"
server_key_iv: "ac6ad193afdab2702f67c5270dfa0d20"
server_key_version: "encryptionkey@0"
I20260929 02:58:52.785423 16816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:58:52.788709 16831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:52.790242 16816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:58:52.790360 16816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "71484ace087340f396044b542cbe3a71"
format_stamp: "Formatted at 2026-09-29 02:58:52 on dist-test-slave-m1rv"
server_key: "1aae2a6c52e6ad765eaa72e2d04fc470"
server_key_iv: "ac6ad193afdab2702f67c5270dfa0d20"
server_key_version: "encryptionkey@0"
I20260929 02:58:52.790596 16816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:52.802405 16816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:52.806239 16816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:52.806452 16816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:52.808692 16816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:52.808807 16816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:52.808966 16816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:52.809046 16816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:58:52 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650732, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
I20260929 02:58:52.832453 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:52.809469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:53083 (local address 127.8.208.126:39247)
0929 02:58:52.809667 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:52.809674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:52.811042 (+ 1368us) server_negotiation.cc:407] Connection header received
0929 02:58:52.813239 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:52.813245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:52.813338 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:52.813470 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:52.815838 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.817256 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.819605 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.820263 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.820425 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:52.825114 (+ 4689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:52.825118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:52.825121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:52.825153 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:52.828447 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:52.829372 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:52.829379 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:52.829382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:52.829527 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:52.830237 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:52.830245 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:52.830251 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:52.830698 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:52.830852 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:52.831865 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0929 02:58:52.832096 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:58:52.833686 16840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:52.810003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39247 (local address 127.8.208.66:53083)
0929 02:58:52.810850 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:52.810906 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:52.812911 (+ 2005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:52.813755 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:52.813773 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:52.814744 (+ 971us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:52.815632 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:52.815663 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.817417 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.817425 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:52.819450 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:52.819460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.820503 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.820512 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:52.820746 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:52.821568 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:52.821602 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:52.824897 (+ 3295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:52.828589 (+ 3692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:52.828601 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:52.828617 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:52.829259 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:52.829689 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:52.829697 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:52.829702 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:52.830048 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:52.830860 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:52.830870 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:52.831373 (+ 503us) client_negotiation.cc:769] Sending connection context
0929 02:58:52.831778 (+ 405us) client_negotiation.cc:240] Negotiation successful
0929 02:58:52.832326 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":394,"threads_started":1}
I20260929 02:58:52.855922 16816 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:39391
I20260929 02:58:52.855978 16950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:39391 every 8 connection(s)
I20260929 02:58:52.857355 16816 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:58:52.859957 16951 heartbeater.cc:347] Connected to a master server at 127.8.208.126:39247
I20260929 02:58:52.860195 16951 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:52.860998 16951 heartbeater.cc:511] Master 127.8.208.126:39247 requested a full tablet report, sending...
I20260929 02:58:52.862569 16604 ts_manager.cc:194] Registered new tserver with Master: 71484ace087340f396044b542cbe3a71 (127.8.208.66:39391)
I20260929 02:58:52.863656 16604 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:53083
I20260929 02:58:52.866673 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:52.852656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39247 (local address 127.8.208.66:49125)
0929 02:58:52.853445 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:52.853472 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:52.853663 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:52.854061 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:52.854066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:52.854519 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:52.854839 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:52.854852 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.856285 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.856291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:52.857401 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:52.857412 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.858437 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.858446 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:52.858636 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:52.859424 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:52.859451 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:52.860384 (+ 933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:52.864154 (+ 3770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:52.864162 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:52.864168 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:52.864652 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:52.865094 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:52.865098 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:52.865100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:52.865267 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:52.865898 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:52.865901 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:52.866038 (+ 137us) client_negotiation.cc:769] Sending connection context
0929 02:58:52.866137 (+ 99us) client_negotiation.cc:240] Negotiation successful
0929 02:58:52.866348 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":367,"threads_started":1}
I20260929 02:58:52.866813 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:52.852666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:49125 (local address 127.8.208.126:39247)
0929 02:58:52.852908 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:52.852916 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:52.853577 (+ 661us) server_negotiation.cc:407] Connection header received
0929 02:58:52.853814 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:52.853821 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:52.853907 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:52.854023 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:52.854992 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.856146 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.857545 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:52.858179 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:52.858351 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:52.860565 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:52.860574 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:52.860580 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:52.860620 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:52.864036 (+ 3416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:52.864812 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:52.864818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:52.864821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:52.864975 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:52.865367 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:52.865372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:52.865376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:52.865788 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:52.865932 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:52.866285 (+ 353us) server_negotiation.cc:299] Negotiation successful
0929 02:58:52.866497 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 02:58:52.867095 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 16816
I20260929 02:58:52.867228 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:58:52.867597 9025 external_mini_cluster.cc:1428] Setting key 3084004678cc875c748058c8fa65ee5a
WARNING: no policy specified for oryx/127.8.208.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab.
Entry for principal oryx/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab.
I20260929 02:58:52.911857 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39247
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab
--principal=oryx/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:39859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:53.242312 16955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:53.242605 16955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:53.242830 16955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:53.250890 16955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:53.251003 16955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:53.251147 16955 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:58:53.251268 16955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:58:53.261052 16955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:39859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab
--principal=oryx/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.208.126:39247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:53.262336 16955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:53.263641 16955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:53.276684 16960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:53.276640 16961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:53.276752 16963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:53.277494 16955 server_base.cc:1034] running on GCE node
I20260929 02:58:53.278069 16955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:53.278975 16955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:53.280208 16955 hybrid_clock.cc:648] HybridClock initialized: now 1790650733280150 us; error 39 us; skew 500 ppm
I20260929 02:58:53.280504 16955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:58:53 dist-test-slave-m1rv krb5kdc[16560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650733, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:58:53.283668 16955 init.cc:377] Logged in from keytab as oryx/127.8.208.67@KRBTEST.COM (short username oryx)
I20260929 02:58:53.285044 16955 webserver.cc:460] Webserver started at http://127.8.208.67:42737/ using document root <none> and password file <none>
I20260929 02:58:53.285722 16955 fs_manager.cc:359] Metadata directory not provided
I20260929 02:58:53.285815 16955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:58:53.286062 16955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:58:53.288755 16955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "e70275b544bc40669454e344d9c92161"
format_stamp: "Formatted at 2026-09-29 02:58:53 on dist-test-slave-m1rv"
server_key: "c415b5d6dc984ee82935dcb3db4e4976"
server_key_iv: "8d84b310bd4ec74c52f5477957b77213"
server_key_version: "encryptionkey@0"
I20260929 02:58:53.289523 16955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "e70275b544bc40669454e344d9c92161"
format_stamp: "Formatted at 2026-09-29 02:58:53 on dist-test-slave-m1rv"
server_key: "c415b5d6dc984ee82935dcb3db4e4976"
server_key_iv: "8d84b310bd4ec74c52f5477957b77213"
server_key_version: "encryptionkey@0"
I20260929 02:58:53.294644 16955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260929 02:58:53.298558 16970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:53.299998 16955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:58:53.300124 16955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "e70275b544bc40669454e344d9c92161"
format_stamp: "Formatted at 2026-09-29 02:58:53 on dist-test-slave-m1rv"
server_key: "c415b5d6dc984ee82935dcb3db4e4976"
server_key_iv: "8d84b310bd4ec74c52f5477957b77213"
server_key_version: "encryptionkey@0"
I20260929 02:58:53.300302 16955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:58:53.316027 16955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:58:53.320380 16955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:58:53.320629 16955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:58:53.323451 16955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:58:53.323559 16955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:58:53.323716 16955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:58:53.323789 16955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:58:53 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650733, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
I20260929 02:58:53.348462 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:53.323976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:47981 (local address 127.8.208.126:39247)
0929 02:58:53.324222 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:53.324230 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:53.325513 (+ 1283us) server_negotiation.cc:407] Connection header received
0929 02:58:53.327718 (+ 2205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:53.327724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:53.327811 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:53.327924 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:53.330301 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:53.331679 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:53.334270 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:53.335109 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:53.335264 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:53.339607 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:53.339616 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:53.339624 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:53.339677 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:53.344334 (+ 4657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:53.345074 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:53.345083 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:53.345088 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:53.345292 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:53.345979 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:53.345987 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:53.345993 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:53.346417 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:53.347375 (+ 958us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:53.347756 (+ 381us) server_negotiation.cc:299] Negotiation successful
0929 02:58:53.348026 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:58:53.350257 16979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:53.324543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39247 (local address 127.8.208.67:47981)
0929 02:58:53.325366 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:53.325401 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:53.327382 (+ 1981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:53.328165 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:53.328173 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:53.328996 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:53.329921 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:53.329983 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:53.331861 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:53.331868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:53.334112 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:53.334127 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:53.335430 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:53.335441 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:53.335794 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:53.336657 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:53.336679 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:53.339354 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:53.344502 (+ 5148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:53.344509 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:53.344521 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:53.344945 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:53.345463 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:53.345469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:53.345473 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:53.345767 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:53.346609 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:53.346621 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:53.347187 (+ 566us) client_negotiation.cc:769] Sending connection context
0929 02:58:53.347563 (+ 376us) client_negotiation.cc:240] Negotiation successful
0929 02:58:53.348267 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":353,"threads_started":1}
I20260929 02:58:53.376320 16812 heartbeater.cc:503] Master 127.8.208.126:39247 was elected leader, sending a full tablet report...
I20260929 02:58:53.379407 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:53.361247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:33997 (local address 127.8.208.126:39247)
0929 02:58:53.361457 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:53.361466 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:53.365793 (+ 4327us) server_negotiation.cc:407] Connection header received
0929 02:58:53.366104 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:53.366113 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:53.366211 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:53.366291 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:53.368108 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:53.368811 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:53.370231 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:53.370626 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:53.370764 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:53.372562 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:53.372567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:53.372570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:53.372597 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:53.375883 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:53.376798 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:53.376806 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:53.376811 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:53.376974 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:53.377525 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:53.377532 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:53.377537 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:53.378024 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:53.378490 (+ 466us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:53.378838 (+ 348us) server_negotiation.cc:299] Negotiation successful
0929 02:58:53.379054 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:58:53.379530 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:53.361227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39247 (local address 127.8.208.67:33997)
0929 02:58:53.365657 (+ 4430us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:53.365680 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:53.365935 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:53.366581 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:53.366586 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:53.367312 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:53.367946 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:53.367966 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:53.368953 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:53.368960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:53.370081 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:53.370092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:53.370812 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:53.370818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:53.370978 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:53.371659 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:53.371676 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:53.372366 (+ 690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:53.376024 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:53.376030 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:53.376033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:53.376618 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:53.377107 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:53.377115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:53.377122 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:53.377351 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:53.378185 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:53.378191 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:53.378541 (+ 350us) client_negotiation.cc:769] Sending connection context
0929 02:58:53.378732 (+ 191us) client_negotiation.cc:240] Negotiation successful
0929 02:58:53.379054 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4279,"thread_start_us":4150,"threads_started":1}
I20260929 02:58:53.387823 16955 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:34131
I20260929 02:58:53.387852 17089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:34131 every 8 connection(s)
I20260929 02:58:53.388985 16955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:58:53.391039 17090 heartbeater.cc:347] Connected to a master server at 127.8.208.126:39247
I20260929 02:58:53.391252 17090 heartbeater.cc:464] Registering TS with master...
I20260929 02:58:53.392062 17090 heartbeater.cc:511] Master 127.8.208.126:39247 requested a full tablet report, sending...
I20260929 02:58:53.392102 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 16955
I20260929 02:58:53.392277 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:58:53.392789 9025 external_mini_cluster.cc:1428] Setting key ee3f9ffcf6b264c2031ff699f164635c
I20260929 02:58:53.393610 16605 ts_manager.cc:194] Registered new tserver with Master: e70275b544bc40669454e344d9c92161 (127.8.208.67:34131)
I20260929 02:58:53.394636 16605 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:47981
I20260929 02:58:53.396051 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:58:53 dist-test-slave-m1rv krb5kdc[16560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650733, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:58:53 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:58:53 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:58:53.866866 16951 heartbeater.cc:503] Master 127.8.208.126:39247 was elected leader, sending a full tablet report...
I20260929 02:58:54.397565 17090 heartbeater.cc:503] Master 127.8.208.126:39247 was elected leader, sending a full tablet report...
W20260929 02:58:54.491225 17098 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.052s user 0.000s sys 0.014s
W20260929 02:58:54.492331 16699 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:58:53.433737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52786 (local address 127.8.208.126:39247)
0929 02:58:53.433917 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:53.433921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:53.434300 (+ 379us) server_negotiation.cc:407] Connection header received
0929 02:58:53.434448 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:53.434452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:53.434512 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:53.434609 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:53.436001 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:53.437108 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:53.438321 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:53.438704 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:53.438796 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.492117 (+1053321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52786: BlockingRecv error: recv got EOF from 127.0.0.1:52786 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 02:58:54 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650733, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
I20260929 02:58:54.510449 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:54.496487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52796 (local address 127.8.208.126:39247)
0929 02:58:54.496717 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:54.496724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:54.496974 (+ 250us) server_negotiation.cc:407] Connection header received
0929 02:58:54.497225 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:54.497229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:54.497303 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:54.497456 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:54.498673 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.499378 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.500637 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.501089 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.501241 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.505620 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:54.505625 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:54.505628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:54.505660 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:54.508217 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:54.508833 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:54.508837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:54.508839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:54.508951 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:54.509349 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:54.509353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:54.509355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:54.509610 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:54.509828 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:54.510058 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 02:58:54.510221 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:58:54.514734 16604 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:58:54.521690 16604 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:54.554302 17112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:54.533691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40940 (local address 127.8.208.65:32855)
0929 02:58:54.545507 (+ 11816us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:54.545522 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:54.545571 (+ 49us) server_negotiation.cc:407] Connection header received
0929 02:58:54.545717 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:54.545730 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:54.546117 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:54.546439 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:54.547433 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.548945 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.551747 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.552967 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.553125 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.553394 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:54.553827 (+ 433us) server_negotiation.cc:299] Negotiation successful
0929 02:58:54.554004 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11252,"thread_start_us":11530,"threads_started":1}
I20260929 02:58:54.554466 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:54.533245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:32855 (local address 127.0.0.1:40940)
0929 02:58:54.534016 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:54.534059 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:54.534250 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:54.546493 (+ 12243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:54.546505 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:54.546558 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:54.547086 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:54.547107 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.549105 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.549112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:54.551526 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:54.551568 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.553221 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.553232 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:54.553508 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.553545 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 02:58:54.553787 (+ 242us) client_negotiation.cc:240] Negotiation successful
0929 02:58:54.553989 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":27,"spinlock_wait_cycles":3456,"thread_start_us":574,"threads_started":1}
I20260929 02:58:54.556296 17111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:54.533576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36904 (local address 127.8.208.66:39391)
0929 02:58:54.547198 (+ 13622us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:54.547226 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:54.547262 (+ 36us) server_negotiation.cc:407] Connection header received
0929 02:58:54.547400 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:54.547412 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:54.547717 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:54.548662 (+ 945us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:54.549309 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.550510 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.553526 (+ 3016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.555210 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.555366 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.555589 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:54.555813 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 02:58:54.555982 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13403,"thread_start_us":13236,"threads_started":1}
I20260929 02:58:54.556970 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:54.533031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:39391 (local address 127.0.0.1:36904)
0929 02:58:54.534016 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:54.534059 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:54.534250 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:54.547976 (+ 13726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:54.547982 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:54.548013 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:54.548999 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:54.549017 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.550658 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.550667 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:54.553309 (+ 2642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:54.553327 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.555425 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.555432 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:54.555604 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.555644 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 02:58:54.555769 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 02:58:54.555906 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":2,"spinlock_wait_cycles":24320,"thread_start_us":701,"threads_started":1}
I20260929 02:58:54.558338 16742 tablet_service.cc:1467] Processing CreateTablet for tablet 23964d19e72a48fd935f1160d8f10096 (DEFAULT_TABLE table=test-table [id=b270af1880964451a1ba7a60a406fa3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:54.559124 17113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:54.534628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:34131 (local address 127.0.0.1:52890)
0929 02:58:54.548040 (+ 13412us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:54.548066 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:54.548216 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:54.548939 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:54.548943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:54.548968 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:54.549440 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:54.549453 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.551319 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.551328 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:54.554967 (+ 3639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:54.554984 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.558251 (+ 3267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.558258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:54.558446 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.558492 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 02:58:54.558600 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 02:58:54.558735 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13257,"mutex_wait_us":505,"thread_start_us":13588,"threads_started":1}
I20260929 02:58:54.559150 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:54.534953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52890 (local address 127.8.208.67:34131)
0929 02:58:54.547989 (+ 13036us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:54.547997 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:54.548203 (+ 206us) server_negotiation.cc:407] Connection header received
0929 02:58:54.548458 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:54.548470 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:54.548727 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:54.548930 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:54.549635 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.551165 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.555132 (+ 3967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.557998 (+ 2866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.558199 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.558420 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:54.558657 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 02:58:54.558840 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12861,"thread_start_us":13007,"threads_started":1}
I20260929 02:58:54.559834 16742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23964d19e72a48fd935f1160d8f10096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:54.560570 16881 tablet_service.cc:1467] Processing CreateTablet for tablet 23964d19e72a48fd935f1160d8f10096 (DEFAULT_TABLE table=test-table [id=b270af1880964451a1ba7a60a406fa3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:54.561863 16881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23964d19e72a48fd935f1160d8f10096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:54.562644 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 23964d19e72a48fd935f1160d8f10096 (DEFAULT_TABLE table=test-table [id=b270af1880964451a1ba7a60a406fa3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:54.563633 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23964d19e72a48fd935f1160d8f10096. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:54.574530 17115 tablet_bootstrap.cc:492] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571: Bootstrap starting.
I20260929 02:58:54.576571 17116 tablet_bootstrap.cc:492] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71: Bootstrap starting.
I20260929 02:58:54.577333 17117 tablet_bootstrap.cc:492] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161: Bootstrap starting.
I20260929 02:58:54.578943 17115 tablet_bootstrap.cc:654] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:54.580255 17115 log.cc:824] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:54.580353 17117 tablet_bootstrap.cc:654] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:54.581326 17117 log.cc:824] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:54.583576 17115 tablet_bootstrap.cc:492] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571: No bootstrap required, opened a new log
I20260929 02:58:54.583873 17115 ts_tablet_manager.cc:1397] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260929 02:58:54.584259 17117 tablet_bootstrap.cc:492] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161: No bootstrap required, opened a new log
I20260929 02:58:54.584268 17116 tablet_bootstrap.cc:654] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:54.584504 17117 ts_tablet_manager.cc:1397] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 02:58:54.585421 17116 log.cc:824] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71: Log is configured to *not* fsync() on all Append() calls
I20260929 02:58:54.588253 17116 tablet_bootstrap.cc:492] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71: No bootstrap required, opened a new log
I20260929 02:58:54.588547 17116 ts_tablet_manager.cc:1397] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.006s
I20260929 02:58:54.593031 17115 raft_consensus.cc:348] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.593573 17115 raft_consensus.cc:374] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:54.593683 17115 raft_consensus.cc:729] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2716754effc24d2594c84fba1b15e571, State: Initialized, Role: FOLLOWER
I20260929 02:58:54.594393 17115 consensus_queue.cc:260] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.594715 17117 raft_consensus.cc:348] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.595276 17117 raft_consensus.cc:374] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:54.595404 17117 raft_consensus.cc:729] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70275b544bc40669454e344d9c92161, State: Initialized, Role: FOLLOWER
I20260929 02:58:54.596019 17117 consensus_queue.cc:260] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.597700 17117 ts_tablet_manager.cc:1428] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260929 02:58:54.598836 17116 raft_consensus.cc:348] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.599252 17116 raft_consensus.cc:374] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:54.599375 17116 raft_consensus.cc:729] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71484ace087340f396044b542cbe3a71, State: Initialized, Role: FOLLOWER
I20260929 02:58:54.599934 17116 consensus_queue.cc:260] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.601558 17116 ts_tablet_manager.cc:1428] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260929 02:58:54.602185 17115 ts_tablet_manager.cc:1428] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571: Time spent starting tablet: real 0.018s user 0.013s sys 0.000s
W20260929 02:58:54.612264 16952 tablet.cc:2369] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:58:54.620195 16813 tablet.cc:2369] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:58:54.631659 17123 raft_consensus.cc:482] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:54.631945 17123 raft_consensus.cc:504] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.633535 17123 leader_election.cc:288] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855), e70275b544bc40669454e344d9c92161 (127.8.208.67:34131)
I20260929 02:58:54.640321 17112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:54.634050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:36521 (local address 127.8.208.65:32855)
0929 02:58:54.634268 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:54.634274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:54.634588 (+ 314us) server_negotiation.cc:407] Connection header received
0929 02:58:54.634812 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:54.634817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:54.634895 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:54.635005 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:54.635689 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.636656 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.638658 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.639369 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.639546 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.639635 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:54.639933 (+ 298us) server_negotiation.cc:299] Negotiation successful
0929 02:58:54.640063 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:58:54.640384 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:54.634085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:51915 (local address 127.8.208.67:34131)
0929 02:58:54.634282 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:54.634288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:54.635037 (+ 749us) server_negotiation.cc:407] Connection header received
0929 02:58:54.635209 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:54.635216 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:54.635292 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:54.635413 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:54.635991 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.636782 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.638618 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.639376 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.639567 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.639641 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:54.639982 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 02:58:54.640101 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 02:58:54.640497 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:54.633767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:32855 (local address 127.8.208.66:36521)
0929 02:58:54.634451 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:54.634473 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:54.634653 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:54.635046 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:54.635051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:54.635087 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:54.635533 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:54.635547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.636784 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.636790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:54.638485 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:54.638496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.639594 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.639600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:54.639775 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.639806 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:58:54.639877 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 02:58:54.640016 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":86,"thread_start_us":309,"threads_started":1}
I20260929 02:58:54.640601 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:54.633767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:34131 (local address 127.8.208.66:51915)
0929 02:58:54.634917 (+ 1150us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:54.634951 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:54.635066 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:54.635431 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:54.635436 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:54.635461 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:54.635848 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:54.635858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.636910 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.636913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:54.638464 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:54.638479 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.639633 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.639639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:54.639804 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.639826 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:58:54.639948 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 02:58:54.640067 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"mutex_wait_us":576,"thread_start_us":298,"threads_started":1}
W20260929 02:58:54.641492 17091 tablet.cc:2369] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:58:54.641665 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23964d19e72a48fd935f1160d8f10096" candidate_uuid: "71484ace087340f396044b542cbe3a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70275b544bc40669454e344d9c92161" is_pre_election: true
I20260929 02:58:54.642202 17041 raft_consensus.cc:2454] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71484ace087340f396044b542cbe3a71 in term 0.
I20260929 02:58:54.642140 16762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23964d19e72a48fd935f1160d8f10096" candidate_uuid: "71484ace087340f396044b542cbe3a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2716754effc24d2594c84fba1b15e571" is_pre_election: true
I20260929 02:58:54.642705 16762 raft_consensus.cc:2454] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71484ace087340f396044b542cbe3a71 in term 0.
I20260929 02:58:54.642875 16835 leader_election.cc:302] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71484ace087340f396044b542cbe3a71, e70275b544bc40669454e344d9c92161; no voters:
I20260929 02:58:54.643296 17123 raft_consensus.cc:2781] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:54.643458 17123 raft_consensus.cc:482] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:54.643571 17123 raft_consensus.cc:3037] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:54.645752 17123 raft_consensus.cc:504] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.646338 17123 leader_election.cc:288] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [CANDIDATE]: Term 1 election: Requested vote from peers 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855), e70275b544bc40669454e344d9c92161 (127.8.208.67:34131)
I20260929 02:58:54.646692 16762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23964d19e72a48fd935f1160d8f10096" candidate_uuid: "71484ace087340f396044b542cbe3a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2716754effc24d2594c84fba1b15e571"
I20260929 02:58:54.646898 16762 raft_consensus.cc:3037] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:54.646852 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23964d19e72a48fd935f1160d8f10096" candidate_uuid: "71484ace087340f396044b542cbe3a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70275b544bc40669454e344d9c92161"
I20260929 02:58:54.647066 17041 raft_consensus.cc:3037] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:54.649296 16762 raft_consensus.cc:2454] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71484ace087340f396044b542cbe3a71 in term 1.
I20260929 02:58:54.649533 17041 raft_consensus.cc:2454] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71484ace087340f396044b542cbe3a71 in term 1.
I20260929 02:58:54.649742 16832 leader_election.cc:302] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2716754effc24d2594c84fba1b15e571, 71484ace087340f396044b542cbe3a71; no voters:
I20260929 02:58:54.650066 17123 raft_consensus.cc:2781] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:54.650516 17123 raft_consensus.cc:686] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 1 LEADER]: Becoming Leader. State: Replica: 71484ace087340f396044b542cbe3a71, State: Running, Role: LEADER
I20260929 02:58:54.650979 17123 consensus_queue.cc:237] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.655377 16603 catalog_manager.cc:5495] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71484ace087340f396044b542cbe3a71 (127.8.208.66). New cstate: current_term: 1 leader_uuid: "71484ace087340f396044b542cbe3a71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } health_report { overall_health: UNKNOWN } } }
I20260929 02:58:54.683809 17111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:54.678984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36908 (local address 127.8.208.66:39391)
0929 02:58:54.679171 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:54.679175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:54.679193 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:58:54.679310 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:54.679314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:54.679384 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:54.679507 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:54.679814 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.680518 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.681849 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:54.682222 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:54.682311 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:54.682691 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:54.683240 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:54.683343 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:54.683452 (+ 109us) server_negotiation.cc:299] Negotiation successful
0929 02:58:54.683513 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:58:54.691747 16762 raft_consensus.cc:1261] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Refusing update from remote peer 71484ace087340f396044b542cbe3a71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:54.691817 17041 raft_consensus.cc:1261] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Refusing update from remote peer 71484ace087340f396044b542cbe3a71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:54.692705 17123 consensus_queue.cc:1035] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:54.692999 17126 consensus_queue.cc:1035] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:54.704243 17129 mvcc.cc:205] Tried to move back new op lower bound from 7334505409287692288 to 7334505409134211072. Current Snapshot: MvccSnapshot[applied={T|T < 7334505409287692288}]
I20260929 02:58:54.713619 17131 mvcc.cc:205] Tried to move back new op lower bound from 7334505409287692288 to 7334505409134211072. Current Snapshot: MvccSnapshot[applied={T|T < 7334505409287692288 or (T in {7334505409287692288})}]
I20260929 02:58:54.718748 16603 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:54.719028 16603 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:54.722402 16603 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822: Sending DeleteTablet for 3 replicas of tablet 23964d19e72a48fd935f1160d8f10096
I20260929 02:58:54.723575 16881 tablet_service.cc:1514] Processing DeleteTablet for tablet 23964d19e72a48fd935f1160d8f10096 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:54 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:36904
I20260929 02:58:54.723841 17020 tablet_service.cc:1514] Processing DeleteTablet for tablet 23964d19e72a48fd935f1160d8f10096 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:54 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:52890
I20260929 02:58:54.723502 16742 tablet_service.cc:1514] Processing DeleteTablet for tablet 23964d19e72a48fd935f1160d8f10096 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:54 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:40940
I20260929 02:58:54.724852 16603 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:58:54.725675 16603 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:54.738067 17019 tablet_service.cc:1467] Processing CreateTablet for tablet 9a4a35406b224473bbde227296af87aa (DEFAULT_TABLE table=test-table [id=08fda14f9ff1441f8750fcf468b0e3a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:54.738430 16741 tablet_service.cc:1467] Processing CreateTablet for tablet 9a4a35406b224473bbde227296af87aa (DEFAULT_TABLE table=test-table [id=08fda14f9ff1441f8750fcf468b0e3a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:54.738679 17019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a4a35406b224473bbde227296af87aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:54.738962 16741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a4a35406b224473bbde227296af87aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:54.741113 16880 tablet_service.cc:1467] Processing CreateTablet for tablet 9a4a35406b224473bbde227296af87aa (DEFAULT_TABLE table=test-table [id=08fda14f9ff1441f8750fcf468b0e3a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:54.741839 16880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a4a35406b224473bbde227296af87aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:54.744414 17140 tablet_replica.cc:330] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571: stopping tablet replica
I20260929 02:58:54.745046 17140 raft_consensus.cc:2229] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:54.746068 17140 raft_consensus.cc:2258] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:54.746702 17138 tablet_replica.cc:330] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71: stopping tablet replica
I20260929 02:58:54.747097 17138 raft_consensus.cc:2229] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:54.747794 17138 raft_consensus.cc:2258] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:54.748216 17115 tablet_bootstrap.cc:492] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571: Bootstrap starting.
I20260929 02:58:54.749974 17115 tablet_bootstrap.cc:654] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:54.750185 17138 ts_tablet_manager.cc:1905] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:54.750775 17140 ts_tablet_manager.cc:1905] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:54.757364 17138 ts_tablet_manager.cc:1918] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:54.757532 17138 log.cc:1196] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/23964d19e72a48fd935f1160d8f10096
I20260929 02:58:54.758038 17138 ts_tablet_manager.cc:1939] T 23964d19e72a48fd935f1160d8f10096 P 71484ace087340f396044b542cbe3a71: Deleting consensus metadata
I20260929 02:58:54.759028 17140 ts_tablet_manager.cc:1918] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:54.759232 17140 log.cc:1196] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/23964d19e72a48fd935f1160d8f10096
I20260929 02:58:54.760030 16592 catalog_manager.cc:4841] TS 71484ace087340f396044b542cbe3a71 (127.8.208.66:39391): tablet 23964d19e72a48fd935f1160d8f10096 (table test-table [id=b270af1880964451a1ba7a60a406fa3e]) successfully deleted
I20260929 02:58:54.760212 17140 ts_tablet_manager.cc:1939] T 23964d19e72a48fd935f1160d8f10096 P 2716754effc24d2594c84fba1b15e571: Deleting consensus metadata
I20260929 02:58:54.761965 16589 catalog_manager.cc:4841] TS 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855): tablet 23964d19e72a48fd935f1160d8f10096 (table test-table [id=b270af1880964451a1ba7a60a406fa3e]) successfully deleted
I20260929 02:58:54.762984 17117 tablet_bootstrap.cc:492] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161: Bootstrap starting.
I20260929 02:58:54.764555 17117 tablet_bootstrap.cc:654] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:54.767021 17116 tablet_bootstrap.cc:492] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71: Bootstrap starting.
I20260929 02:58:54.768596 17116 tablet_bootstrap.cc:654] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:54.776533 17115 tablet_bootstrap.cc:492] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571: No bootstrap required, opened a new log
I20260929 02:58:54.776685 17115 ts_tablet_manager.cc:1397] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.004s
I20260929 02:58:54.777472 17115 raft_consensus.cc:348] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.777705 17115 raft_consensus.cc:374] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:54.777781 17115 raft_consensus.cc:729] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2716754effc24d2594c84fba1b15e571, State: Initialized, Role: FOLLOWER
I20260929 02:58:54.778072 17115 consensus_queue.cc:260] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.780807 17115 ts_tablet_manager.cc:1428] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260929 02:58:54.783242 17139 tablet_replica.cc:330] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161: stopping tablet replica
I20260929 02:58:54.783716 17139 raft_consensus.cc:2229] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:54.784212 17139 raft_consensus.cc:2258] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:54.786372 17139 ts_tablet_manager.cc:1905] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:54.786777 17116 tablet_bootstrap.cc:492] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71: No bootstrap required, opened a new log
I20260929 02:58:54.786953 17116 ts_tablet_manager.cc:1397] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.005s
I20260929 02:58:54.787912 17116 raft_consensus.cc:348] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.788184 17116 raft_consensus.cc:374] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:54.788317 17116 raft_consensus.cc:729] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71484ace087340f396044b542cbe3a71, State: Initialized, Role: FOLLOWER
I20260929 02:58:54.788630 17116 consensus_queue.cc:260] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.789381 17116 ts_tablet_manager.cc:1428] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260929 02:58:54.790740 17117 tablet_bootstrap.cc:492] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161: No bootstrap required, opened a new log
I20260929 02:58:54.790934 17117 ts_tablet_manager.cc:1397] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.009s
I20260929 02:58:54.791757 17117 raft_consensus.cc:348] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.791994 17117 raft_consensus.cc:374] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:54.792101 17117 raft_consensus.cc:729] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70275b544bc40669454e344d9c92161, State: Initialized, Role: FOLLOWER
I20260929 02:58:54.792472 17117 consensus_queue.cc:260] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.793788 17117 ts_tablet_manager.cc:1428] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260929 02:58:54.795063 17139 ts_tablet_manager.cc:1918] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:54.795228 17139 log.cc:1196] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/23964d19e72a48fd935f1160d8f10096
I20260929 02:58:54.795823 17139 ts_tablet_manager.cc:1939] T 23964d19e72a48fd935f1160d8f10096 P e70275b544bc40669454e344d9c92161: Deleting consensus metadata
I20260929 02:58:54.797590 16592 catalog_manager.cc:4841] TS e70275b544bc40669454e344d9c92161 (127.8.208.67:34131): tablet 23964d19e72a48fd935f1160d8f10096 (table test-table [id=b270af1880964451a1ba7a60a406fa3e]) successfully deleted
I20260929 02:58:54.960290 17123 raft_consensus.cc:482] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:54.960471 17123 raft_consensus.cc:504] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.961222 17123 leader_election.cc:288] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855), e70275b544bc40669454e344d9c92161 (127.8.208.67:34131)
I20260929 02:58:54.961818 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a4a35406b224473bbde227296af87aa" candidate_uuid: "71484ace087340f396044b542cbe3a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70275b544bc40669454e344d9c92161" is_pre_election: true
I20260929 02:58:54.961826 16762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a4a35406b224473bbde227296af87aa" candidate_uuid: "71484ace087340f396044b542cbe3a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2716754effc24d2594c84fba1b15e571" is_pre_election: true
I20260929 02:58:54.962085 17041 raft_consensus.cc:2454] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71484ace087340f396044b542cbe3a71 in term 0.
I20260929 02:58:54.962100 16762 raft_consensus.cc:2454] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71484ace087340f396044b542cbe3a71 in term 0.
I20260929 02:58:54.962539 16832 leader_election.cc:302] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2716754effc24d2594c84fba1b15e571, 71484ace087340f396044b542cbe3a71; no voters:
I20260929 02:58:54.962815 17123 raft_consensus.cc:2781] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:54.962975 17123 raft_consensus.cc:482] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:54.963104 17123 raft_consensus.cc:3037] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:54.965365 17123 raft_consensus.cc:504] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.966020 17123 leader_election.cc:288] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [CANDIDATE]: Term 1 election: Requested vote from peers 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855), e70275b544bc40669454e344d9c92161 (127.8.208.67:34131)
I20260929 02:58:54.966529 16762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a4a35406b224473bbde227296af87aa" candidate_uuid: "71484ace087340f396044b542cbe3a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2716754effc24d2594c84fba1b15e571"
I20260929 02:58:54.966632 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a4a35406b224473bbde227296af87aa" candidate_uuid: "71484ace087340f396044b542cbe3a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70275b544bc40669454e344d9c92161"
I20260929 02:58:54.966734 16762 raft_consensus.cc:3037] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:54.966835 17041 raft_consensus.cc:3037] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:54.969691 17041 raft_consensus.cc:2454] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71484ace087340f396044b542cbe3a71 in term 1.
I20260929 02:58:54.969691 16762 raft_consensus.cc:2454] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71484ace087340f396044b542cbe3a71 in term 1.
I20260929 02:58:54.970115 16832 leader_election.cc:302] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2716754effc24d2594c84fba1b15e571, 71484ace087340f396044b542cbe3a71; no voters:
I20260929 02:58:54.970393 17123 raft_consensus.cc:2781] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:54.970551 17123 raft_consensus.cc:686] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 1 LEADER]: Becoming Leader. State: Replica: 71484ace087340f396044b542cbe3a71, State: Running, Role: LEADER
I20260929 02:58:54.970829 17123 consensus_queue.cc:237] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:54.973929 16603 catalog_manager.cc:5495] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71484ace087340f396044b542cbe3a71 (127.8.208.66). New cstate: current_term: 1 leader_uuid: "71484ace087340f396044b542cbe3a71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } health_report { overall_health: UNKNOWN } } }
Sep 29 02:58:55 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650731, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.126@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
I20260929 02:58:55.043970 17150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:55.027569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52812 (local address 127.8.208.126:39247)
0929 02:58:55.028574 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:55.028580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:55.028993 (+ 413us) server_negotiation.cc:407] Connection header received
0929 02:58:55.029237 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:55.029241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:55.029296 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:55.029403 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:55.030995 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.031799 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.033798 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.034195 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.034294 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.038316 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:55.038322 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:55.038325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:55.038352 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:55.041114 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:55.042067 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:55.042071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:55.042073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:55.042202 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:55.042619 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:55.042622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:55.042624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:55.042873 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:55.042989 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:55.043504 (+ 515us) server_negotiation.cc:299] Negotiation successful
0929 02:58:55.043685 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":841,"thread_start_us":637,"threads_started":1}
I20260929 02:58:55.044341 17151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:55.027533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39247 (local address 127.0.0.1:52812)
0929 02:58:55.028856 (+ 1323us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:55.028876 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:55.029055 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:55.029461 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:55.029466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:55.030480 (+ 1014us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:58:55.030808 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:55.030821 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.031954 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.031963 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:55.033658 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:55.033673 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.034381 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.034388 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:55.034562 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.035344 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:58:55.035369 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:58:55.038104 (+ 2735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:58:55.041333 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:55.041346 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:55.041354 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:55.041949 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:55.042320 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:58:55.042326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:58:55.042329 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:58:55.042512 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:58:55.043012 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:58:55.043024 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:58:55.043226 (+ 202us) client_negotiation.cc:769] Sending connection context
0929 02:58:55.043440 (+ 214us) client_negotiation.cc:240] Negotiation successful
0929 02:58:55.043862 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1179,"mutex_wait_us":113,"thread_start_us":562,"threads_started":1}
I20260929 02:58:55.049795 16603 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:52812:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 02:58:55.050870 16603 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:55.057902 16741 tablet_service.cc:1467] Processing CreateTablet for tablet 66bd75471a934a5285cd555bc9403748 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9bf67ed80db949aead377a5c43c50dc2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:58:55.058477 16741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66bd75471a934a5285cd555bc9403748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:55.058290 16880 tablet_service.cc:1467] Processing CreateTablet for tablet 66bd75471a934a5285cd555bc9403748 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9bf67ed80db949aead377a5c43c50dc2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:58:55.058820 16880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66bd75471a934a5285cd555bc9403748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:55.060751 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 66bd75471a934a5285cd555bc9403748 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9bf67ed80db949aead377a5c43c50dc2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:58:55.061307 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66bd75471a934a5285cd555bc9403748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:55.065675 17115 tablet_bootstrap.cc:492] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571: Bootstrap starting.
I20260929 02:58:55.066557 17116 tablet_bootstrap.cc:492] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: Bootstrap starting.
I20260929 02:58:55.067366 17115 tablet_bootstrap.cc:654] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:55.068203 17116 tablet_bootstrap.cc:654] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:55.069392 17117 tablet_bootstrap.cc:492] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161: Bootstrap starting.
I20260929 02:58:55.071033 17117 tablet_bootstrap.cc:654] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:55.074776 17117 tablet_bootstrap.cc:492] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161: No bootstrap required, opened a new log
I20260929 02:58:55.075026 17117 ts_tablet_manager.cc:1397] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260929 02:58:55.075789 17117 raft_consensus.cc:348] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.076104 17117 raft_consensus.cc:374] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:55.076222 17117 raft_consensus.cc:729] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70275b544bc40669454e344d9c92161, State: Initialized, Role: FOLLOWER
I20260929 02:58:55.076557 17117 consensus_queue.cc:260] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.077025 17122 tablet_replica.cc:439] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } } }
I20260929 02:58:55.077452 17122 tablet_replica.cc:442] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:55.077914 17117 ts_tablet_manager.cc:1428] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260929 02:58:55.079061 17115 tablet_bootstrap.cc:492] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571: No bootstrap required, opened a new log
I20260929 02:58:55.079238 17115 ts_tablet_manager.cc:1397] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260929 02:58:55.079981 17115 raft_consensus.cc:348] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.080227 17115 raft_consensus.cc:374] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:55.080304 17115 raft_consensus.cc:729] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2716754effc24d2594c84fba1b15e571, State: Initialized, Role: FOLLOWER
I20260929 02:58:55.080582 17115 consensus_queue.cc:260] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.081053 17121 tablet_replica.cc:439] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } } }
I20260929 02:58:55.081432 17121 tablet_replica.cc:442] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:55.081887 17115 ts_tablet_manager.cc:1428] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 02:58:55.086491 17116 tablet_bootstrap.cc:492] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: No bootstrap required, opened a new log
I20260929 02:58:55.086678 17116 ts_tablet_manager.cc:1397] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.005s
I20260929 02:58:55.087457 17116 raft_consensus.cc:348] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.087736 17116 raft_consensus.cc:374] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:55.087827 17116 raft_consensus.cc:729] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71484ace087340f396044b542cbe3a71, State: Initialized, Role: FOLLOWER
I20260929 02:58:55.088110 17116 consensus_queue.cc:260] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.088835 17116 ts_tablet_manager.cc:1428] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260929 02:58:55.088699 17123 tablet_replica.cc:439] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } } }
I20260929 02:58:55.089049 17123 tablet_replica.cc:442] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:55.345973 17123 raft_consensus.cc:482] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:55.346208 17123 raft_consensus.cc:504] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.346812 17123 leader_election.cc:288] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855), e70275b544bc40669454e344d9c92161 (127.8.208.67:34131)
I20260929 02:58:55.347405 16762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66bd75471a934a5285cd555bc9403748" candidate_uuid: "71484ace087340f396044b542cbe3a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2716754effc24d2594c84fba1b15e571" is_pre_election: true
I20260929 02:58:55.347410 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66bd75471a934a5285cd555bc9403748" candidate_uuid: "71484ace087340f396044b542cbe3a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70275b544bc40669454e344d9c92161" is_pre_election: true
I20260929 02:58:55.347648 16762 raft_consensus.cc:2454] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71484ace087340f396044b542cbe3a71 in term 0.
I20260929 02:58:55.347683 17041 raft_consensus.cc:2454] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71484ace087340f396044b542cbe3a71 in term 0.
I20260929 02:58:55.348093 16832 leader_election.cc:302] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2716754effc24d2594c84fba1b15e571, 71484ace087340f396044b542cbe3a71; no voters:
I20260929 02:58:55.348366 17123 raft_consensus.cc:2781] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:55.348455 17123 raft_consensus.cc:482] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:55.348526 17123 raft_consensus.cc:3037] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:55.350385 17123 raft_consensus.cc:504] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.350916 17123 leader_election.cc:288] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [CANDIDATE]: Term 1 election: Requested vote from peers 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855), e70275b544bc40669454e344d9c92161 (127.8.208.67:34131)
I20260929 02:58:55.351274 16762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66bd75471a934a5285cd555bc9403748" candidate_uuid: "71484ace087340f396044b542cbe3a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2716754effc24d2594c84fba1b15e571"
I20260929 02:58:55.351316 17041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66bd75471a934a5285cd555bc9403748" candidate_uuid: "71484ace087340f396044b542cbe3a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70275b544bc40669454e344d9c92161"
I20260929 02:58:55.351490 16762 raft_consensus.cc:3037] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:55.351523 17041 raft_consensus.cc:3037] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:55.354278 17041 raft_consensus.cc:2454] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71484ace087340f396044b542cbe3a71 in term 1.
I20260929 02:58:55.354278 16762 raft_consensus.cc:2454] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71484ace087340f396044b542cbe3a71 in term 1.
I20260929 02:58:55.354676 16835 leader_election.cc:302] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2716754effc24d2594c84fba1b15e571, 71484ace087340f396044b542cbe3a71, e70275b544bc40669454e344d9c92161; no voters:
I20260929 02:58:55.354950 17123 raft_consensus.cc:2781] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:55.355146 17123 raft_consensus.cc:686] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [term 1 LEADER]: Becoming Leader. State: Replica: 71484ace087340f396044b542cbe3a71, State: Running, Role: LEADER
I20260929 02:58:55.355398 17123 consensus_queue.cc:237] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.356839 17126 tablet_replica.cc:439] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: TxnStatusTablet state changed. Reason: New leader 71484ace087340f396044b542cbe3a71. Latest consensus state: current_term: 1 leader_uuid: "71484ace087340f396044b542cbe3a71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } } }
I20260929 02:58:55.357108 17126 tablet_replica.cc:442] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:58:55.357671 17156 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 02:58:55.357894 17156 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 02:58:55.358390 16604 catalog_manager.cc:5495] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71484ace087340f396044b542cbe3a71 (127.8.208.66). New cstate: current_term: 1 leader_uuid: "71484ace087340f396044b542cbe3a71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } health_report { overall_health: UNKNOWN } } }
I20260929 02:58:55.387771 17151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:55.380702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:39391 (local address 127.0.0.1:36910)
0929 02:58:55.380907 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:55.380937 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:55.381060 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:55.382051 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:55.382056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:55.382071 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:58:55.382274 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:55.382285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.383822 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.383827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:55.385505 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:55.385519 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.386524 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.386532 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:55.386712 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.386780 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:58:55.387230 (+ 450us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:58:55.387250 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 02:58:55.387334 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 02:58:55.387497 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 02:58:55.388440 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:55.380809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36910 (local address 127.8.208.66:39391)
0929 02:58:55.381696 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:55.381702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:55.381721 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:58:55.381798 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:55.381803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:55.381880 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:55.381987 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:55.382459 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.383673 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.385661 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.386355 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.386478 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.386979 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:55.387123 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:55.387208 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:55.388002 (+ 794us) server_negotiation.cc:299] Negotiation successful
0929 02:58:55.388071 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":616,"threads_started":1}
I20260929 02:58:55.394137 17041 raft_consensus.cc:1261] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Refusing update from remote peer 71484ace087340f396044b542cbe3a71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:55.394186 16762 raft_consensus.cc:1261] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Refusing update from remote peer 71484ace087340f396044b542cbe3a71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:55.394829 17123 consensus_queue.cc:1035] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:55.395094 17126 consensus_queue.cc:1035] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:55.416265 17121 tablet_replica.cc:439] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571: TxnStatusTablet state changed. Reason: New leader 71484ace087340f396044b542cbe3a71. Latest consensus state: current_term: 1 leader_uuid: "71484ace087340f396044b542cbe3a71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } } }
I20260929 02:58:55.416662 17121 tablet_replica.cc:442] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:55.419697 17123 tablet_replica.cc:439] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "71484ace087340f396044b542cbe3a71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } } }
I20260929 02:58:55.420011 17123 tablet_replica.cc:442] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:58:55.422616 17122 tablet_replica.cc:439] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161: TxnStatusTablet state changed. Reason: New leader 71484ace087340f396044b542cbe3a71. Latest consensus state: current_term: 1 leader_uuid: "71484ace087340f396044b542cbe3a71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } } }
I20260929 02:58:55.422880 17122 tablet_replica.cc:442] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:55.422865 17121 tablet_replica.cc:439] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "71484ace087340f396044b542cbe3a71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } } }
I20260929 02:58:55.423121 17121 tablet_replica.cc:442] T 66bd75471a934a5285cd555bc9403748 P 2716754effc24d2594c84fba1b15e571: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:55.423888 17126 tablet_replica.cc:439] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "71484ace087340f396044b542cbe3a71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } } }
I20260929 02:58:55.424131 17126 tablet_replica.cc:442] T 66bd75471a934a5285cd555bc9403748 P 71484ace087340f396044b542cbe3a71: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:58:55.427237 17122 tablet_replica.cc:439] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "71484ace087340f396044b542cbe3a71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } } }
I20260929 02:58:55.427479 17122 tablet_replica.cc:442] T 66bd75471a934a5285cd555bc9403748 P e70275b544bc40669454e344d9c92161: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:58:55.444165 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:55.438395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:51899 (local address 127.8.208.66:39391)
0929 02:58:55.438634 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:55.438643 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:55.439189 (+ 546us) server_negotiation.cc:407] Connection header received
0929 02:58:55.439382 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:55.439386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:55.439449 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:55.439537 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:55.440037 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.440876 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.442471 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.442904 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.443040 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.443439 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:55.443534 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:55.443622 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:55.443757 (+ 135us) server_negotiation.cc:299] Negotiation successful
0929 02:58:55.443819 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":79}
I20260929 02:58:55.444284 17170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:55.438322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:39391 (local address 127.8.208.66:51899)
0929 02:58:55.439091 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:55.439106 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:55.439264 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:55.439556 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:55.439560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:55.439571 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:58:55.439882 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:55.439890 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.441006 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.441013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:55.442325 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:55.442337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.443074 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.443080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:55.443223 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.443269 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:58:55.443620 (+ 351us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:58:55.443630 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 02:58:55.443728 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 02:58:55.443851 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":523,"threads_started":1}
I20260929 02:58:55.453020 16761 raft_consensus.cc:1261] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Refusing update from remote peer 71484ace087340f396044b542cbe3a71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:55.453516 17126 consensus_queue.cc:1035] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:55.453797 17171 consensus_queue.cc:1035] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:55.471244 17162 mvcc.cc:205] Tried to move back new op lower bound from 7334505412411461632 to 7334505410443558912. Current Snapshot: MvccSnapshot[applied={T|T < 7334505412411461632}]
I20260929 02:58:55.509446 16605 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:55.509654 16605 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:55.512466 16605 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822: Sending DeleteTablet for 3 replicas of tablet 9a4a35406b224473bbde227296af87aa
I20260929 02:58:55.513252 16741 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a4a35406b224473bbde227296af87aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:55 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:40940
I20260929 02:58:55.513900 17020 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a4a35406b224473bbde227296af87aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:55 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:52890
I20260929 02:58:55.513981 17175 tablet_replica.cc:330] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571: stopping tablet replica
I20260929 02:58:55.514222 16881 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a4a35406b224473bbde227296af87aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:55 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:36904
I20260929 02:58:55.514250 17175 raft_consensus.cc:2229] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:55.514564 17175 raft_consensus.cc:2258] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:55.515709 17175 ts_tablet_manager.cc:1905] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:55.523452 17176 tablet_replica.cc:330] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161: stopping tablet replica
I20260929 02:58:55.523766 17176 raft_consensus.cc:2229] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:55.524077 17176 raft_consensus.cc:2258] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:55.524051 17175 ts_tablet_manager.cc:1918] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:55.524197 17175 log.cc:1196] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/9a4a35406b224473bbde227296af87aa
I20260929 02:58:55.524830 17175 ts_tablet_manager.cc:1939] T 9a4a35406b224473bbde227296af87aa P 2716754effc24d2594c84fba1b15e571: Deleting consensus metadata
I20260929 02:58:55.525236 17176 ts_tablet_manager.cc:1905] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:55.526146 16589 catalog_manager.cc:4841] TS 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855): tablet 9a4a35406b224473bbde227296af87aa (table test-table [id=08fda14f9ff1441f8750fcf468b0e3a0]) successfully deleted
I20260929 02:58:55.530758 17177 tablet_replica.cc:330] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71: stopping tablet replica
I20260929 02:58:55.531054 17177 raft_consensus.cc:2229] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:55.531585 17177 raft_consensus.cc:2258] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:55.532439 17176 ts_tablet_manager.cc:1918] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:55.532621 17176 log.cc:1196] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/9a4a35406b224473bbde227296af87aa
I20260929 02:58:55.532769 17177 ts_tablet_manager.cc:1905] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:55.533165 17176 ts_tablet_manager.cc:1939] T 9a4a35406b224473bbde227296af87aa P e70275b544bc40669454e344d9c92161: Deleting consensus metadata
I20260929 02:58:55.534358 16592 catalog_manager.cc:4841] TS e70275b544bc40669454e344d9c92161 (127.8.208.67:34131): tablet 9a4a35406b224473bbde227296af87aa (table test-table [id=08fda14f9ff1441f8750fcf468b0e3a0]) successfully deleted
I20260929 02:58:55.541280 17177 ts_tablet_manager.cc:1918] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:55.541455 17177 log.cc:1196] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/9a4a35406b224473bbde227296af87aa
I20260929 02:58:55.542016 17177 ts_tablet_manager.cc:1939] T 9a4a35406b224473bbde227296af87aa P 71484ace087340f396044b542cbe3a71: Deleting consensus metadata
I20260929 02:58:55.543040 16592 catalog_manager.cc:4841] TS 71484ace087340f396044b542cbe3a71 (127.8.208.66:39391): tablet 9a4a35406b224473bbde227296af87aa (table test-table [id=08fda14f9ff1441f8750fcf468b0e3a0]) successfully deleted
Sep 29 02:58:55 dist-test-slave-m1rv krb5kdc[16560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:58:55 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:58:55 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:58:55.587137 17184 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:58:55.570172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52824 (local address 127.8.208.126:39247)
0929 02:58:55.570665 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:55.570670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:55.571110 (+ 440us) server_negotiation.cc:407] Connection header received
0929 02:58:55.571317 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:55.571327 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:55.571390 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:55.571512 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:55.572773 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.573552 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.574976 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.575358 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.575495 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.586805 (+ 11310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52824: BlockingRecv error: recv got EOF from 127.0.0.1:52824 (error 108)
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":253,"threads_started":1}
Sep 29 02:58:55 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM
I20260929 02:58:55.604058 17184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:55.589929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52832 (local address 127.8.208.126:39247)
0929 02:58:55.590077 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:55.590081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:55.590738 (+ 657us) server_negotiation.cc:407] Connection header received
0929 02:58:55.590942 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:55.590949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:55.591017 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:55.591120 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:55.592397 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.593278 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.594644 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.595042 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.595204 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.598824 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:55.598832 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:55.598841 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:55.598870 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:55.601504 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:55.602168 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:55.602173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:55.602177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:55.602306 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:55.602669 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:55.602675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:55.602679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:55.602941 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:55.603199 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:55.603462 (+ 263us) server_negotiation.cc:299] Negotiation successful
0929 02:58:55.603683 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:58:55.607501 16605 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:58:55.608124 16605 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:55.615157 16881 tablet_service.cc:1467] Processing CreateTablet for tablet 9a721544de0447748700d2fb5bc5d922 (DEFAULT_TABLE table=test-table [id=8f679ff59917405f8e4a81dd86c0cf4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:55.615574 16881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a721544de0447748700d2fb5bc5d922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:55.617632 17020 tablet_service.cc:1467] Processing CreateTablet for tablet 9a721544de0447748700d2fb5bc5d922 (DEFAULT_TABLE table=test-table [id=8f679ff59917405f8e4a81dd86c0cf4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:55.618037 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a721544de0447748700d2fb5bc5d922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:55.623075 16741 tablet_service.cc:1467] Processing CreateTablet for tablet 9a721544de0447748700d2fb5bc5d922 (DEFAULT_TABLE table=test-table [id=8f679ff59917405f8e4a81dd86c0cf4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:55.623651 16741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a721544de0447748700d2fb5bc5d922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:55.630240 17191 tablet_bootstrap.cc:492] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71: Bootstrap starting.
I20260929 02:58:55.632709 17191 tablet_bootstrap.cc:654] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:55.633087 17192 tablet_bootstrap.cc:492] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161: Bootstrap starting.
I20260929 02:58:55.635907 17192 tablet_bootstrap.cc:654] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:55.636507 17193 tablet_bootstrap.cc:492] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571: Bootstrap starting.
I20260929 02:58:55.639000 17193 tablet_bootstrap.cc:654] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:55.639305 17191 tablet_bootstrap.cc:492] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71: No bootstrap required, opened a new log
I20260929 02:58:55.639468 17191 ts_tablet_manager.cc:1397] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260929 02:58:55.640339 17192 tablet_bootstrap.cc:492] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161: No bootstrap required, opened a new log
I20260929 02:58:55.640601 17192 ts_tablet_manager.cc:1397] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260929 02:58:55.640412 17191 raft_consensus.cc:348] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.640784 17191 raft_consensus.cc:374] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:55.640897 17191 raft_consensus.cc:729] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71484ace087340f396044b542cbe3a71, State: Initialized, Role: FOLLOWER
I20260929 02:58:55.641225 17191 consensus_queue.cc:260] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.643275 17192 raft_consensus.cc:348] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.643683 17192 raft_consensus.cc:374] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:55.643891 17192 raft_consensus.cc:729] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70275b544bc40669454e344d9c92161, State: Initialized, Role: FOLLOWER
I20260929 02:58:55.644248 17192 consensus_queue.cc:260] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.645164 17192 ts_tablet_manager.cc:1428] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260929 02:58:55.645678 17191 ts_tablet_manager.cc:1428] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260929 02:58:55.647181 17193 tablet_bootstrap.cc:492] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571: No bootstrap required, opened a new log
I20260929 02:58:55.647320 17193 ts_tablet_manager.cc:1397] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260929 02:58:55.648100 17193 raft_consensus.cc:348] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.648342 17193 raft_consensus.cc:374] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:55.648440 17193 raft_consensus.cc:729] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2716754effc24d2594c84fba1b15e571, State: Initialized, Role: FOLLOWER
I20260929 02:58:55.648684 17193 consensus_queue.cc:260] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.649782 17193 ts_tablet_manager.cc:1428] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260929 02:58:55.859473 17122 raft_consensus.cc:482] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:55.859782 17122 raft_consensus.cc:504] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.861851 17122 leader_election.cc:288] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855), 71484ace087340f396044b542cbe3a71 (127.8.208.66:39391)
I20260929 02:58:55.873900 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:55.862231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:60975 (local address 127.8.208.66:39391)
0929 02:58:55.862488 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:55.862497 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:55.863054 (+ 557us) server_negotiation.cc:407] Connection header received
0929 02:58:55.863332 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:55.863338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:55.863406 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:55.863460 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:55.864250 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.865410 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.868379 (+ 2969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.869728 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.869846 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.869918 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:55.870474 (+ 556us) server_negotiation.cc:299] Negotiation successful
0929 02:58:55.870614 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260929 02:58:55.874881 17197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:55.862119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:39391 (local address 127.8.208.67:60975)
0929 02:58:55.862947 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:55.862971 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:55.863158 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:55.863631 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:55.863636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:55.863658 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:55.864074 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:55.864085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.865592 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.865600 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:55.868201 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:55.868224 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.870095 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.870103 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:55.870298 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.870331 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:58:55.874305 (+ 3974us) client_negotiation.cc:240] Negotiation successful
0929 02:58:55.874468 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":558,"threads_started":1}
I20260929 02:58:55.876153 16901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a721544de0447748700d2fb5bc5d922" candidate_uuid: "e70275b544bc40669454e344d9c92161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71484ace087340f396044b542cbe3a71" is_pre_election: true
I20260929 02:58:55.876617 16901 raft_consensus.cc:2454] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e70275b544bc40669454e344d9c92161 in term 0.
I20260929 02:58:55.878628 16974 leader_election.cc:302] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71484ace087340f396044b542cbe3a71, e70275b544bc40669454e344d9c92161; no voters:
I20260929 02:58:55.879034 17122 raft_consensus.cc:2781] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:55.879170 17122 raft_consensus.cc:482] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:55.879274 17122 raft_consensus.cc:3037] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:55.882016 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:55.862156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:32855 (local address 127.8.208.67:55837)
0929 02:58:55.873154 (+ 10998us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:55.873199 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:55.873337 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:55.874710 (+ 1373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:55.874715 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:55.874744 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:58:55.875320 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:55.875335 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.877386 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.877396 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:55.879601 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:55.879617 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.881208 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.881215 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:55.881384 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.881418 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:58:55.881515 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 02:58:55.881633 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10833,"thread_start_us":10683,"threads_started":1}
I20260929 02:58:55.881973 17122 raft_consensus.cc:504] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.882046 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:55.862327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:55837 (local address 127.8.208.65:32855)
0929 02:58:55.873860 (+ 11533us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:55.873897 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:55.873920 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:58:55.874020 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:55.874025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:55.874163 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:55.874306 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:58:55.875621 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.877150 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.879751 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.880978 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.881158 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.881273 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:55.881567 (+ 294us) server_negotiation.cc:299] Negotiation successful
0929 02:58:55.881724 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11355,"thread_start_us":11167,"threads_started":1}
I20260929 02:58:55.882889 17122 leader_election.cc:288] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [CANDIDATE]: Term 1 election: Requested vote from peers 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855), 71484ace087340f396044b542cbe3a71 (127.8.208.66:39391)
I20260929 02:58:55.882850 16761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a721544de0447748700d2fb5bc5d922" candidate_uuid: "e70275b544bc40669454e344d9c92161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2716754effc24d2594c84fba1b15e571" is_pre_election: true
I20260929 02:58:55.883124 16761 raft_consensus.cc:2454] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e70275b544bc40669454e344d9c92161 in term 0.
I20260929 02:58:55.883191 16901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a721544de0447748700d2fb5bc5d922" candidate_uuid: "e70275b544bc40669454e344d9c92161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71484ace087340f396044b542cbe3a71"
I20260929 02:58:55.883451 16901 raft_consensus.cc:3037] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:55.883661 16761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a721544de0447748700d2fb5bc5d922" candidate_uuid: "e70275b544bc40669454e344d9c92161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2716754effc24d2594c84fba1b15e571"
I20260929 02:58:55.883826 16761 raft_consensus.cc:3037] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:55.885885 16901 raft_consensus.cc:2454] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e70275b544bc40669454e344d9c92161 in term 1.
I20260929 02:58:55.886267 16974 leader_election.cc:302] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71484ace087340f396044b542cbe3a71, e70275b544bc40669454e344d9c92161; no voters:
I20260929 02:58:55.886531 17122 raft_consensus.cc:2781] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:55.886654 16761 raft_consensus.cc:2454] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e70275b544bc40669454e344d9c92161 in term 1.
I20260929 02:58:55.887113 17122 raft_consensus.cc:686] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 1 LEADER]: Becoming Leader. State: Replica: e70275b544bc40669454e344d9c92161, State: Running, Role: LEADER
I20260929 02:58:55.887497 17122 consensus_queue.cc:237] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.891276 16605 catalog_manager.cc:5495] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 reported cstate change: term changed from 0 to 1, leader changed from <none> to e70275b544bc40669454e344d9c92161 (127.8.208.67). New cstate: current_term: 1 leader_uuid: "e70275b544bc40669454e344d9c92161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } health_report { overall_health: HEALTHY } } }
I20260929 02:58:55.928766 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:55.923195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52894 (local address 127.8.208.67:34131)
0929 02:58:55.923802 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:55.923806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:55.923842 (+ 36us) server_negotiation.cc:407] Connection header received
0929 02:58:55.923925 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:55.923929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:55.924016 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:55.924161 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:55.924547 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.925319 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.926595 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:55.927030 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:55.927142 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:55.927503 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:55.928121 (+ 618us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:55.928234 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:55.928317 (+ 83us) server_negotiation.cc:299] Negotiation successful
0929 02:58:55.928388 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":370,"threads_started":1}
I20260929 02:58:55.932425 16761 raft_consensus.cc:1261] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Refusing update from remote peer e70275b544bc40669454e344d9c92161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:55.932622 16901 raft_consensus.cc:1261] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Refusing update from remote peer e70275b544bc40669454e344d9c92161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:55.933202 17200 consensus_queue.cc:1035] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:55.933591 17122 consensus_queue.cc:1035] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:55.952602 16605 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:55.952920 16605 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:55.956084 16605 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822: Sending DeleteTablet for 3 replicas of tablet 9a721544de0447748700d2fb5bc5d922
I20260929 02:58:55.957058 17020 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a721544de0447748700d2fb5bc5d922 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:55 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:52890
I20260929 02:58:55.957037 16741 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a721544de0447748700d2fb5bc5d922 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:55 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:40940
I20260929 02:58:55.957418 17176 tablet_replica.cc:330] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161: stopping tablet replica
I20260929 02:58:55.957471 17175 tablet_replica.cc:330] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571: stopping tablet replica
I20260929 02:58:55.957651 17176 raft_consensus.cc:2229] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:55.957672 17175 raft_consensus.cc:2229] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:55.957907 17175 raft_consensus.cc:2258] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:55.958218 17176 raft_consensus.cc:2258] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:55.958745 16881 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a721544de0447748700d2fb5bc5d922 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:55 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:36904
I20260929 02:58:55.959040 17177 tablet_replica.cc:330] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71: stopping tablet replica
I20260929 02:58:55.959256 17177 raft_consensus.cc:2229] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:55.959481 17177 raft_consensus.cc:2258] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:55.959700 17176 ts_tablet_manager.cc:1905] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:55.960477 17177 ts_tablet_manager.cc:1905] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:55.964701 16605 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 02:58:55.965030 17175 ts_tablet_manager.cc:1905] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 02:58:55.965469 16605 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:58:55.967834 17177 ts_tablet_manager.cc:1918] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:55.967978 17177 log.cc:1196] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/9a721544de0447748700d2fb5bc5d922
I20260929 02:58:55.968393 17177 ts_tablet_manager.cc:1939] T 9a721544de0447748700d2fb5bc5d922 P 71484ace087340f396044b542cbe3a71: Deleting consensus metadata
I20260929 02:58:55.969483 16592 catalog_manager.cc:4841] TS 71484ace087340f396044b542cbe3a71 (127.8.208.66:39391): tablet 9a721544de0447748700d2fb5bc5d922 (table test-table [id=8f679ff59917405f8e4a81dd86c0cf4b]) successfully deleted
I20260929 02:58:55.972314 17175 ts_tablet_manager.cc:1918] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:55.972494 17175 log.cc:1196] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/9a721544de0447748700d2fb5bc5d922
I20260929 02:58:55.972993 17175 ts_tablet_manager.cc:1939] T 9a721544de0447748700d2fb5bc5d922 P 2716754effc24d2594c84fba1b15e571: Deleting consensus metadata
I20260929 02:58:55.974009 16881 tablet_service.cc:1467] Processing CreateTablet for tablet c2e30030c3b24da993f7ce49eb63231b (DEFAULT_TABLE table=test-table [id=9a6b5ed8d9b74cec93177c2449a1e3b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:55.974300 16741 tablet_service.cc:1467] Processing CreateTablet for tablet c2e30030c3b24da993f7ce49eb63231b (DEFAULT_TABLE table=test-table [id=9a6b5ed8d9b74cec93177c2449a1e3b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:55.974452 16881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e30030c3b24da993f7ce49eb63231b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:55.974812 16741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e30030c3b24da993f7ce49eb63231b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:55.979146 17191 tablet_bootstrap.cc:492] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71: Bootstrap starting.
I20260929 02:58:55.980041 17020 tablet_service.cc:1467] Processing CreateTablet for tablet c2e30030c3b24da993f7ce49eb63231b (DEFAULT_TABLE table=test-table [id=9a6b5ed8d9b74cec93177c2449a1e3b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:58:55.980495 17020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2e30030c3b24da993f7ce49eb63231b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:58:55.980720 17191 tablet_bootstrap.cc:654] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:55.980923 17176 ts_tablet_manager.cc:1918] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:58:55.981077 17176 log.cc:1196] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/9a721544de0447748700d2fb5bc5d922
I20260929 02:58:55.981592 17176 ts_tablet_manager.cc:1939] T 9a721544de0447748700d2fb5bc5d922 P e70275b544bc40669454e344d9c92161: Deleting consensus metadata
I20260929 02:58:55.982115 16589 catalog_manager.cc:4841] TS 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855): tablet 9a721544de0447748700d2fb5bc5d922 (table test-table [id=8f679ff59917405f8e4a81dd86c0cf4b]) successfully deleted
I20260929 02:58:55.982864 16592 catalog_manager.cc:4841] TS e70275b544bc40669454e344d9c92161 (127.8.208.67:34131): tablet 9a721544de0447748700d2fb5bc5d922 (table test-table [id=8f679ff59917405f8e4a81dd86c0cf4b]) successfully deleted
I20260929 02:58:55.983278 17193 tablet_bootstrap.cc:492] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571: Bootstrap starting.
I20260929 02:58:55.985502 17193 tablet_bootstrap.cc:654] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:55.986541 17192 tablet_bootstrap.cc:492] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161: Bootstrap starting.
I20260929 02:58:55.988622 17192 tablet_bootstrap.cc:654] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161: Neither blocks nor log segments found. Creating new log.
I20260929 02:58:55.997881 17192 tablet_bootstrap.cc:492] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161: No bootstrap required, opened a new log
I20260929 02:58:55.998102 17192 ts_tablet_manager.cc:1397] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260929 02:58:55.998934 17191 tablet_bootstrap.cc:492] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71: No bootstrap required, opened a new log
I20260929 02:58:55.999225 17191 ts_tablet_manager.cc:1397] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.005s
I20260929 02:58:55.999023 17192 raft_consensus.cc:348] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:55.999353 17192 raft_consensus.cc:374] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:55.999459 17192 raft_consensus.cc:729] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70275b544bc40669454e344d9c92161, State: Initialized, Role: FOLLOWER
I20260929 02:58:55.999776 17192 consensus_queue.cc:260] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:56.000531 17191 raft_consensus.cc:348] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:56.000947 17191 raft_consensus.cc:374] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:56.001091 17191 raft_consensus.cc:729] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71484ace087340f396044b542cbe3a71, State: Initialized, Role: FOLLOWER
I20260929 02:58:56.001608 17192 ts_tablet_manager.cc:1428] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 02:58:56.002491 17191 consensus_queue.cc:260] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:56.003278 17193 tablet_bootstrap.cc:492] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571: No bootstrap required, opened a new log
I20260929 02:58:56.003391 17191 ts_tablet_manager.cc:1428] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260929 02:58:56.003477 17193 ts_tablet_manager.cc:1397] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.007s
I20260929 02:58:56.004436 17193 raft_consensus.cc:348] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:56.004997 17193 raft_consensus.cc:374] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:58:56.005103 17193 raft_consensus.cc:729] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2716754effc24d2594c84fba1b15e571, State: Initialized, Role: FOLLOWER
I20260929 02:58:56.005431 17193 consensus_queue.cc:260] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:56.006671 17193 ts_tablet_manager.cc:1428] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 02:58:56.118647 17206 raft_consensus.cc:482] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:58:56.118938 17206 raft_consensus.cc:504] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:56.119782 17206 leader_election.cc:288] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855), 71484ace087340f396044b542cbe3a71 (127.8.208.66:39391)
I20260929 02:58:56.120316 16761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e30030c3b24da993f7ce49eb63231b" candidate_uuid: "e70275b544bc40669454e344d9c92161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2716754effc24d2594c84fba1b15e571" is_pre_election: true
I20260929 02:58:56.120349 16901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e30030c3b24da993f7ce49eb63231b" candidate_uuid: "e70275b544bc40669454e344d9c92161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71484ace087340f396044b542cbe3a71" is_pre_election: true
I20260929 02:58:56.120586 16901 raft_consensus.cc:2454] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e70275b544bc40669454e344d9c92161 in term 0.
I20260929 02:58:56.120604 16761 raft_consensus.cc:2454] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e70275b544bc40669454e344d9c92161 in term 0.
I20260929 02:58:56.121033 16974 leader_election.cc:302] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71484ace087340f396044b542cbe3a71, e70275b544bc40669454e344d9c92161; no voters:
I20260929 02:58:56.121374 17206 raft_consensus.cc:2781] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:58:56.121519 17206 raft_consensus.cc:482] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:58:56.121637 17206 raft_consensus.cc:3037] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:56.123971 17206 raft_consensus.cc:504] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:56.124521 17206 leader_election.cc:288] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [CANDIDATE]: Term 1 election: Requested vote from peers 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855), 71484ace087340f396044b542cbe3a71 (127.8.208.66:39391)
I20260929 02:58:56.124986 16761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e30030c3b24da993f7ce49eb63231b" candidate_uuid: "e70275b544bc40669454e344d9c92161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2716754effc24d2594c84fba1b15e571"
I20260929 02:58:56.125005 16901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2e30030c3b24da993f7ce49eb63231b" candidate_uuid: "e70275b544bc40669454e344d9c92161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71484ace087340f396044b542cbe3a71"
I20260929 02:58:56.125227 16901 raft_consensus.cc:3037] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:56.125236 16761 raft_consensus.cc:3037] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:58:56.128101 16761 raft_consensus.cc:2454] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e70275b544bc40669454e344d9c92161 in term 1.
I20260929 02:58:56.128118 16901 raft_consensus.cc:2454] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e70275b544bc40669454e344d9c92161 in term 1.
I20260929 02:58:56.128494 16974 leader_election.cc:302] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71484ace087340f396044b542cbe3a71, e70275b544bc40669454e344d9c92161; no voters:
I20260929 02:58:56.128826 17206 raft_consensus.cc:2781] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:58:56.128990 17206 raft_consensus.cc:686] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 1 LEADER]: Becoming Leader. State: Replica: e70275b544bc40669454e344d9c92161, State: Running, Role: LEADER
I20260929 02:58:56.129338 17206 consensus_queue.cc:237] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } }
I20260929 02:58:56.132103 16603 catalog_manager.cc:5495] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 reported cstate change: term changed from 0 to 1, leader changed from <none> to e70275b544bc40669454e344d9c92161 (127.8.208.67). New cstate: current_term: 1 leader_uuid: "e70275b544bc40669454e344d9c92161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e70275b544bc40669454e344d9c92161" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 34131 } health_report { overall_health: HEALTHY } } }
I20260929 02:58:56.167176 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:56.161316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:56351 (local address 127.8.208.66:39391)
0929 02:58:56.161531 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:56.161537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:56.161881 (+ 344us) server_negotiation.cc:407] Connection header received
0929 02:58:56.162057 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:56.162061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:56.162124 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:56.162199 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:56.162783 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.163461 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.165552 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.165946 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.166038 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:56.166596 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:56.166694 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:56.166761 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:56.166910 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 02:58:56.166970 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:58:56.167455 17217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:56.161135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:39391 (local address 127.8.208.67:56351)
0929 02:58:56.161773 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:56.161795 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:56.161937 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:56.162248 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:56.162252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:56.162264 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:58:56.162639 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:56.162651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.163595 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.163600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:56.165415 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:56.165430 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.166131 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.166139 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:56.166345 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:56.166417 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:58:56.166789 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:58:56.166802 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 02:58:56.166887 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 02:58:56.167027 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":281,"threads_started":1}
I20260929 02:58:56.175177 16901 raft_consensus.cc:1261] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Refusing update from remote peer e70275b544bc40669454e344d9c92161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:56.175261 16761 raft_consensus.cc:1261] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Refusing update from remote peer e70275b544bc40669454e344d9c92161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:58:56.175873 17122 consensus_queue.cc:1035] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716754effc24d2594c84fba1b15e571" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:56.176656 17122 consensus_queue.cc:1035] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71484ace087340f396044b542cbe3a71" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:58:56.219322 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:56.212892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:34131 (local address 127.8.208.66:58139)
0929 02:58:56.213598 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0929 02:58:56.213620 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:58:56.213752 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:58:56.214097 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:58:56.214102 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:58:56.214115 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:58:56.214376 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:56.214390 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.215440 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.215445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:56.216985 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:58:56.217001 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.218101 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.218106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:58:56.218315 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:56.218382 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:58:56.218792 (+ 410us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:58:56.218806 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 02:58:56.218884 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 02:58:56.219013 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":417,"threads_started":1}
I20260929 02:58:56.219332 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:56.212997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:58139 (local address 127.8.208.67:34131)
0929 02:58:56.213270 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:56.213280 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:56.213693 (+ 413us) server_negotiation.cc:407] Connection header received
0929 02:58:56.213880 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:56.213885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:56.213976 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:56.214063 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:58:56.214529 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.215302 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.217146 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.217814 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.217972 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:56.218568 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:58:56.218685 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:58:56.218789 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:56.218908 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 02:58:56.218992 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:58:56.256565 16604 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:56.256852 16604 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:58:56.259999 16604 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1327141dd1f4ed399c610d0cd1a3822: Sending DeleteTablet for 3 replicas of tablet c2e30030c3b24da993f7ce49eb63231b
I20260929 02:58:56.260819 16741 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e30030c3b24da993f7ce49eb63231b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:56 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:40940
I20260929 02:58:56.261037 17020 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e30030c3b24da993f7ce49eb63231b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:56 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:52890
I20260929 02:58:56.261063 16881 tablet_service.cc:1514] Processing DeleteTablet for tablet c2e30030c3b24da993f7ce49eb63231b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:58:56 UTC) from {username='oryx', principal='oryx/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:36904
I20260929 02:58:56.261132 17175 tablet_replica.cc:330] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571: stopping tablet replica
I20260929 02:58:56.261335 17175 raft_consensus.cc:2229] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:56.261377 17176 tablet_replica.cc:330] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161: stopping tablet replica
I20260929 02:58:56.261385 17177 tablet_replica.cc:330] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71: stopping tablet replica
I20260929 02:58:56.261540 17177 raft_consensus.cc:2229] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:58:56.261557 17175 raft_consensus.cc:2258] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:56.261560 17176 raft_consensus.cc:2229] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:58:56.261732 17177 raft_consensus.cc:2258] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:56.261904 17176 raft_consensus.cc:2258] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:58:56.262861 17177 ts_tablet_manager.cc:1905] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:56.263378 17175 ts_tablet_manager.cc:1905] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:56.263933 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 16676
I20260929 02:58:56.267217 17176 ts_tablet_manager.cc:1905] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:58:56.271361 17177 ts_tablet_manager.cc:1918] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:56.271520 17177 log.cc:1196] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/c2e30030c3b24da993f7ce49eb63231b
I20260929 02:58:56.272070 17177 ts_tablet_manager.cc:1939] T c2e30030c3b24da993f7ce49eb63231b P 71484ace087340f396044b542cbe3a71: Deleting consensus metadata
I20260929 02:58:56.273459 16592 catalog_manager.cc:4841] TS 71484ace087340f396044b542cbe3a71 (127.8.208.66:39391): tablet c2e30030c3b24da993f7ce49eb63231b (table test-table [id=9a6b5ed8d9b74cec93177c2449a1e3b7]) successfully deleted
I20260929 02:58:56.277197 17176 ts_tablet_manager.cc:1918] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:56.277272 17175 ts_tablet_manager.cc:1918] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:58:56.277371 17176 log.cc:1196] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/c2e30030c3b24da993f7ce49eb63231b
I20260929 02:58:56.277558 17175 log.cc:1196] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/c2e30030c3b24da993f7ce49eb63231b
I20260929 02:58:56.277896 17176 ts_tablet_manager.cc:1939] T c2e30030c3b24da993f7ce49eb63231b P e70275b544bc40669454e344d9c92161: Deleting consensus metadata
I20260929 02:58:56.278085 17175 ts_tablet_manager.cc:1939] T c2e30030c3b24da993f7ce49eb63231b P 2716754effc24d2594c84fba1b15e571: Deleting consensus metadata
I20260929 02:58:56.278982 16592 catalog_manager.cc:4841] TS e70275b544bc40669454e344d9c92161 (127.8.208.67:34131): tablet c2e30030c3b24da993f7ce49eb63231b (table test-table [id=9a6b5ed8d9b74cec93177c2449a1e3b7]) successfully deleted
I20260929 02:58:56.279119 16589 catalog_manager.cc:4841] TS 2716754effc24d2594c84fba1b15e571 (127.8.208.65:32855): tablet c2e30030c3b24da993f7ce49eb63231b (table test-table [id=9a6b5ed8d9b74cec93177c2449a1e3b7]) successfully deleted
Sep 29 02:58:56 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.208.65@KRBTEST.COM
I20260929 02:58:56.290441 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:56.266022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40944 (local address 127.8.208.65:32855)
0929 02:58:56.266201 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:56.266208 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:56.273657 (+ 7449us) server_negotiation.cc:407] Connection header received
0929 02:58:56.273976 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:56.273982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:56.274058 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:56.274135 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:56.275830 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.276950 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.279263 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.279730 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.279803 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:56.284437 (+ 4634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:56.284451 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:56.284468 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:56.284501 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:56.287396 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:56.288186 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:56.288193 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:56.288195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:56.288320 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:56.288733 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:56.288736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:56.288738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:56.289242 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:56.289499 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:56.289818 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 02:58:56.289995 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:58:56.291131 16807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:56.394711 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 16676
I20260929 02:58:56.419808 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 16816
Sep 29 02:58:56 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.208.66@KRBTEST.COM
I20260929 02:58:56.436908 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:56.420626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36922 (local address 127.8.208.66:39391)
0929 02:58:56.420802 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:56.420808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:56.420861 (+ 53us) server_negotiation.cc:407] Connection header received
0929 02:58:56.421030 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:56.421036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:56.421123 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:56.421273 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:56.422387 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.423473 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.424714 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.425539 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.425659 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:56.429894 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:56.429909 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:56.429927 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:56.429965 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:56.433601 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:56.434212 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:56.434221 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:56.434224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:56.434419 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:56.434792 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:56.434800 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:56.434806 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:56.435339 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:56.435642 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:56.436226 (+ 584us) server_negotiation.cc:299] Negotiation successful
0929 02:58:56.436562 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:58:56.437690 16946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:56.584218 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 16816
I20260929 02:58:56.611707 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 16955
Sep 29 02:58:56 dist-test-slave-m1rv krb5kdc[16560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.208.67@KRBTEST.COM
I20260929 02:58:56.626021 17203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:58:56.612435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52900 (local address 127.8.208.67:34131)
0929 02:58:56.612638 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 02:58:56.612642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:58:56.612662 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:58:56.612793 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:58:56.612796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:58:56.612863 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:58:56.612963 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:58:56.614317 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.615216 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.616389 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:58:56.616777 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:58:56.616867 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:58:56.620929 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:58:56.620943 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:58:56.620959 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:58:56.620994 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:58:56.623831 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:56.624391 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:56.624397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:56.624399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:56.624528 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:58:56.624824 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:58:56.624827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:58:56.624829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:58:56.625139 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:58:56.625490 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0929 02:58:56.625621 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 02:58:56.625773 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:58:56.626657 17085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:56.752442 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 16955
I20260929 02:58:56.775143 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 16573
I20260929 02:58:56.776211 16645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:58:56.889904 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 16573
2026-09-29T02:58:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6885 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:58:56 dist-test-slave-m1rv krb5kdc[17230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:58:56 dist-test-slave-m1rv krb5kdc[17230](info): set up 2 sockets
Sep 29 02:58:56 dist-test-slave-m1rv krb5kdc[17230](info): commencing operation
krb5kdc: starting...
W20260929 02:58:58.086925 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.003s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:58:58 dist-test-slave-m1rv krb5kdc[17230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:58:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:58:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:58:58.302977 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:40813
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:40813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:58:58.622113 17243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:58:58.622396 17243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:58:58.622494 17243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:58:58.629563 17243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:58:58.629673 17243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:58:58.629719 17243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:58:58.629777 17243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:58:58.629828 17243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:58:58.639331 17243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:40813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:40813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:58:58.640545 17243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:58:58.641769 17243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:58:58.653752 17248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:58.653687 17251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:58:58.653718 17249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:58:58.655150 17243 server_base.cc:1034] running on GCE node
I20260929 02:58:58.655908 17243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:58:58.657358 17243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:58:58.658599 17243 hybrid_clock.cc:648] HybridClock initialized: now 1790650738658566 us; error 46 us; skew 500 ppm
I20260929 02:58:58.658947 17243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T02:58:58Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1841 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:58:58 dist-test-slave-m1rv krb5kdc[17257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:58:58 dist-test-slave-m1rv krb5kdc[17257](info): set up 2 sockets
Sep 29 02:58:58 dist-test-slave-m1rv krb5kdc[17257](info): commencing operation
krb5kdc: starting...
W20260929 02:58:59.846213 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:58:59 dist-test-slave-m1rv krb5kdc[17257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:59:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:59:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:59:00.070060 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37527
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:37527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:59:00.386932 17270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:00.387223 17270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:00.387423 17270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:00.394649 17270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:59:00.394795 17270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:00.394881 17270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:00.394984 17270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:59:00.395052 17270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:59:00.404843 17270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:37527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:00.406248 17270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:00.407545 17270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:00.419123 17278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:00.419157 17275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:00.419157 17276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:00.420091 17270 server_base.cc:1034] running on GCE node
I20260929 02:59:00.420909 17270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:00.422425 17270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:00.423640 17270 hybrid_clock.cc:648] HybridClock initialized: now 1790650740423590 us; error 43 us; skew 500 ppm
I20260929 02:59:00.424007 17270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:59:00 dist-test-slave-m1rv krb5kdc[17257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650740, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:59:00.427188 17270 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:59:00.428668 17270 webserver.cc:460] Webserver started at http://127.8.208.126:33213/ using document root <none> and password file <none>
I20260929 02:59:00.429422 17270 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:00.429549 17270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:00.429862 17270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:00.432669 17270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "8e5581e4e3134dae8b28a7a80810fdb8"
format_stamp: "Formatted at 2026-09-29 02:59:00 on dist-test-slave-m1rv"
server_key: "cedee417b4273144680aceb3bf7ae3dd"
server_key_iv: "349ab32eac3c5a963ffb3c547acf747e"
server_key_version: "encryptionkey@0"
I20260929 02:59:00.433465 17270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "8e5581e4e3134dae8b28a7a80810fdb8"
format_stamp: "Formatted at 2026-09-29 02:59:00 on dist-test-slave-m1rv"
server_key: "cedee417b4273144680aceb3bf7ae3dd"
server_key_iv: "349ab32eac3c5a963ffb3c547acf747e"
server_key_version: "encryptionkey@0"
I20260929 02:59:00.438452 17270 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260929 02:59:00.441751 17285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:00.443143 17270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 02:59:00.443328 17270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "8e5581e4e3134dae8b28a7a80810fdb8"
format_stamp: "Formatted at 2026-09-29 02:59:00 on dist-test-slave-m1rv"
server_key: "cedee417b4273144680aceb3bf7ae3dd"
server_key_iv: "349ab32eac3c5a963ffb3c547acf747e"
server_key_version: "encryptionkey@0"
I20260929 02:59:00.443526 17270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:00.456272 17270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:00.460527 17270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:00.460758 17270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:00.478883 17270 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:37527
I20260929 02:59:00.478902 17336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:37527 every 8 connection(s)
I20260929 02:59:00.480249 17270 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:59:00.480702 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 17270
I20260929 02:59:00.480856 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:59:00.481706 9025 external_mini_cluster.cc:1428] Setting key e4f4ce3d9e0d1b6e4220e4999550c9f7
I20260929 02:59:00.484814 17337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:59:00 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:59:00.497284 17337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8: Bootstrap starting.
I20260929 02:59:00.501371 17337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8: Neither blocks nor log segments found. Creating new log.
I20260929 02:59:00.502552 17337 log.cc:824] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8: Log is configured to *not* fsync() on all Append() calls
I20260929 02:59:00.505934 17337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8: No bootstrap required, opened a new log
I20260929 02:59:00.506544 17340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:00.483730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52706 (local address 127.8.208.126:37527)
0929 02:59:00.484541 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:00.484557 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:00.484607 (+ 50us) server_negotiation.cc:407] Connection header received
0929 02:59:00.486939 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:00.486966 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:00.487426 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:00.487870 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:00.489157 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:00.491295 (+ 2138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:00.492536 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:00.493587 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:00.493782 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:00.497956 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:00.497966 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:00.497983 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:00.498026 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:00.501983 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:00.502681 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:00.502689 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:00.502697 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:00.502941 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:00.503391 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:00.503394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:00.503396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:00.504018 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:00.504295 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:00.504779 (+ 484us) server_negotiation.cc:299] Negotiation successful
0929 02:59:00.505241 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":336,"threads_started":1}
I20260929 02:59:00.514089 17337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37527 } }
I20260929 02:59:00.514424 17337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:59:00.514498 17337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e5581e4e3134dae8b28a7a80810fdb8, State: Initialized, Role: FOLLOWER
I20260929 02:59:00.514958 17337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37527 } }
I20260929 02:59:00.515152 17337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:59:00.515314 17337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:59:00.515484 17337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:59:00.517606 17337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37527 } }
I20260929 02:59:00.518028 17337 leader_election.cc:302] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e5581e4e3134dae8b28a7a80810fdb8; no voters:
I20260929 02:59:00.518729 17337 leader_election.cc:288] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:59:00.518844 17342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:59:00.519474 17342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 1 LEADER]: Becoming Leader. State: Replica: 8e5581e4e3134dae8b28a7a80810fdb8, State: Running, Role: LEADER
I20260929 02:59:00.519996 17342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37527 } }
I20260929 02:59:00.520880 17337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:59:00.522864 17344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e5581e4e3134dae8b28a7a80810fdb8. Latest consensus state: current_term: 1 leader_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37527 } } }
I20260929 02:59:00.523247 17344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [sys.catalog]: This master's current role is: LEADER
I20260929 02:59:00.523080 17343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37527 } } }
I20260929 02:59:00.523463 17343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [sys.catalog]: This master's current role is: LEADER
I20260929 02:59:00.527812 17349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:59:00.532922 17349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:59:00.541931 17349 catalog_manager.cc:1347] Generated new cluster ID: bcc622fc52b241d581ac5c238937e13a
I20260929 02:59:00.542070 17349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:59:00.555748 17349 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:59:00.557056 17349 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:59:00.568198 17349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8: Generated new TSK 0
I20260929 02:59:00.569034 17349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 02:59:00.634455 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37527
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:59:00.963811 17363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:00.964097 17363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:00.964267 17363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:00.971663 17363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:00.971812 17363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:00.971995 17363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:59:00.982120 17363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:00.983441 17363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:00.984746 17363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:00.998189 17369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:00.998189 17368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:00.998236 17371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:00.998716 17363 server_base.cc:1034] running on GCE node
I20260929 02:59:00.999447 17363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:01.000360 17363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:01.001557 17363 hybrid_clock.cc:648] HybridClock initialized: now 1790650741001517 us; error 36 us; skew 500 ppm
I20260929 02:59:01.001865 17363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:59:01 dist-test-slave-m1rv krb5kdc[17257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650741, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:59:01.011864 17363 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 02:59:01.013877 17363 webserver.cc:460] Webserver started at http://127.8.208.65:33743/ using document root <none> and password file <none>
I20260929 02:59:01.014747 17363 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:01.014904 17363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:01.015235 17363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:01.018256 17363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "0e1b15b9ceb4434b9bd241998f0b0f84"
format_stamp: "Formatted at 2026-09-29 02:59:01 on dist-test-slave-m1rv"
server_key: "c0cb292555e6d7aa79b9eb7689781f2b"
server_key_iv: "097a3b9cae75b7d3f3a69877958443a5"
server_key_version: "encryptionkey@0"
I20260929 02:59:01.019029 17363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "0e1b15b9ceb4434b9bd241998f0b0f84"
format_stamp: "Formatted at 2026-09-29 02:59:01 on dist-test-slave-m1rv"
server_key: "c0cb292555e6d7aa79b9eb7689781f2b"
server_key_iv: "097a3b9cae75b7d3f3a69877958443a5"
server_key_version: "encryptionkey@0"
I20260929 02:59:01.024286 17363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:59:01.027609 17378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:01.029007 17363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:59:01.029162 17363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "0e1b15b9ceb4434b9bd241998f0b0f84"
format_stamp: "Formatted at 2026-09-29 02:59:01 on dist-test-slave-m1rv"
server_key: "c0cb292555e6d7aa79b9eb7689781f2b"
server_key_iv: "097a3b9cae75b7d3f3a69877958443a5"
server_key_version: "encryptionkey@0"
I20260929 02:59:01.029434 17363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:01.041682 17363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:01.045563 17363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:01.045908 17363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:01.047142 17363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:01.048575 17363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:01.048643 17363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:01.048775 17363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:01.048832 17363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:01.081684 17363 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:46731
I20260929 02:59:01.081733 17490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:46731 every 8 connection(s)
I20260929 02:59:01.082998 17363 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:59:01.085704 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 17363
I20260929 02:59:01.085928 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:59:01.086670 9025 external_mini_cluster.cc:1428] Setting key eae1030f7fccfd805393c15ca3523501
Sep 29 02:59:01 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650741, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:59:01.105022 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:01.085580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:35997 (local address 127.8.208.126:37527)
0929 02:59:01.086093 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:01.086100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:01.087213 (+ 1113us) server_negotiation.cc:407] Connection header received
0929 02:59:01.089524 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:01.089533 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:01.089619 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:01.089740 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:01.091289 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:01.092413 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:01.094053 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:01.094798 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:01.094943 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:01.099118 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:01.099123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:01.099127 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:01.099158 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:01.101971 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:01.102701 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:01.102706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:01.102715 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:01.102829 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:01.103275 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:01.103280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:01.103284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:01.103550 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:01.103720 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:01.104427 (+ 707us) server_negotiation.cc:299] Negotiation successful
0929 02:59:01.104662 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":243,"threads_started":1}
I20260929 02:59:01.106118 17494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:01.086221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:35997)
0929 02:59:01.087026 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:01.087075 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:01.089242 (+ 2167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:01.090009 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:01.090018 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:01.090541 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:01.091111 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:01.091128 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:01.092595 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:01.092598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:01.093905 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:01.093921 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:01.095009 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:01.095013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:01.095362 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:01.096111 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:01.096144 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:01.098878 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:59:01.102118 (+ 3240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:59:01.102127 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:59:01.102145 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:59:01.102567 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:59:01.102951 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:59:01.102956 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:59:01.102959 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:59:01.103148 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:59:01.103681 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:59:01.103692 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:59:01.104047 (+ 355us) client_negotiation.cc:769] Sending connection context
0929 02:59:01.104320 (+ 273us) client_negotiation.cc:240] Negotiation successful
0929 02:59:01.104849 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":298,"threads_started":1}
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
I20260929 02:59:01.107813 17491 heartbeater.cc:347] Connected to a master server at 127.8.208.126:37527
I20260929 02:59:01.108132 17491 heartbeater.cc:464] Registering TS with master...
I20260929 02:59:01.108824 17491 heartbeater.cc:511] Master 127.8.208.126:37527 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
I20260929 02:59:01.111166 17302 ts_manager.cc:194] Registered new tserver with Master: 0e1b15b9ceb4434b9bd241998f0b0f84 (127.8.208.65:46731)
I20260929 02:59:01.113353 17302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:35997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 02:59:01.138074 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37527
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:59:01.479800 17497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:01.480087 17497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:01.480263 17497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:01.488454 17497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:01.488557 17497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:01.488744 17497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:59:01.498415 17497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:01.499655 17497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:01.501096 17497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:01.514153 17502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:01.514123 17505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:01.514374 17503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:01.514878 17497 server_base.cc:1034] running on GCE node
I20260929 02:59:01.515488 17497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:01.516415 17497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:01.517669 17497 hybrid_clock.cc:648] HybridClock initialized: now 1790650741517624 us; error 59 us; skew 500 ppm
I20260929 02:59:01.518031 17497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:59:01 dist-test-slave-m1rv krb5kdc[17257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650741, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:59:01.521206 17497 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 02:59:01.522753 17497 webserver.cc:460] Webserver started at http://127.8.208.66:37633/ using document root <none> and password file <none>
I20260929 02:59:01.523452 17497 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:01.523597 17497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:01.523902 17497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:01.526963 17497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "3c122f464f414d59aba99c60329ef286"
format_stamp: "Formatted at 2026-09-29 02:59:01 on dist-test-slave-m1rv"
server_key: "145ed309d1519f8e1ecd4b0222dfcba0"
server_key_iv: "ff3dbe0a24b680d1c8cdf9264257a594"
server_key_version: "encryptionkey@0"
I20260929 02:59:01.527863 17497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "3c122f464f414d59aba99c60329ef286"
format_stamp: "Formatted at 2026-09-29 02:59:01 on dist-test-slave-m1rv"
server_key: "145ed309d1519f8e1ecd4b0222dfcba0"
server_key_iv: "ff3dbe0a24b680d1c8cdf9264257a594"
server_key_version: "encryptionkey@0"
I20260929 02:59:01.533010 17497 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 02:59:01.536536 17512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:01.537853 17497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:59:01.538065 17497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "3c122f464f414d59aba99c60329ef286"
format_stamp: "Formatted at 2026-09-29 02:59:01 on dist-test-slave-m1rv"
server_key: "145ed309d1519f8e1ecd4b0222dfcba0"
server_key_iv: "ff3dbe0a24b680d1c8cdf9264257a594"
server_key_version: "encryptionkey@0"
I20260929 02:59:01.538290 17497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:01.569471 17497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:01.574156 17497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:01.574460 17497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:01.575429 17497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:01.576762 17497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:01.576874 17497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:01.577028 17497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:01.577113 17497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:01.609954 17497 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:40847
I20260929 02:59:01.610034 17624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:40847 every 8 connection(s)
I20260929 02:59:01.611711 17497 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:59:01.619690 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 17497
I20260929 02:59:01.619858 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:59:01.620267 9025 external_mini_cluster.cc:1428] Setting key 3e74f923fb7bb5a434e7612808f5e18a
Sep 29 02:59:01 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650741, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:59:01.633786 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:01.614637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:54115 (local address 127.8.208.126:37527)
0929 02:59:01.615092 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:01.615100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:01.616428 (+ 1328us) server_negotiation.cc:407] Connection header received
0929 02:59:01.619183 (+ 2755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:01.619189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:01.619275 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:01.619407 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:01.621132 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:01.621955 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:01.623561 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:01.624004 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:01.624143 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:01.627724 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:01.627730 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:01.627735 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:01.627772 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:01.630590 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:01.631507 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:01.631511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:01.631513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:01.631630 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:01.632111 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:01.632115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:01.632117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:01.632350 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:01.632455 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:01.633365 (+ 910us) server_negotiation.cc:299] Negotiation successful
0929 02:59:01.633536 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":244,"threads_started":1}
I20260929 02:59:01.635324 17628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:01.615303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:54115)
0929 02:59:01.616256 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:01.616313 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:01.618898 (+ 2585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:01.619648 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:01.619657 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:01.620264 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:01.620965 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:01.620993 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:01.622120 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:01.622125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:01.623412 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:01.623424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:01.624241 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:01.624247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:01.624488 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:01.625244 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:01.625267 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:01.627479 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:59:01.630732 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:59:01.630743 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:59:01.630764 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:59:01.631388 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:59:01.631735 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:59:01.631743 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:59:01.631749 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:59:01.632003 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:59:01.632479 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:59:01.632490 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:59:01.632959 (+ 469us) client_negotiation.cc:769] Sending connection context
0929 02:59:01.633287 (+ 328us) client_negotiation.cc:240] Negotiation successful
0929 02:59:01.633850 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":408,"threads_started":1}
I20260929 02:59:01.636787 17625 heartbeater.cc:347] Connected to a master server at 127.8.208.126:37527
I20260929 02:59:01.637099 17625 heartbeater.cc:464] Registering TS with master...
I20260929 02:59:01.637864 17625 heartbeater.cc:511] Master 127.8.208.126:37527 requested a full tablet report, sending...
I20260929 02:59:01.639357 17302 ts_manager.cc:194] Registered new tserver with Master: 3c122f464f414d59aba99c60329ef286 (127.8.208.66:40847)
I20260929 02:59:01.640164 17302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:54115
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 02:59:01.673516 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37527
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:59:02.005403 17631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:02.005704 17631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:02.005805 17631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:02.013264 17631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:02.013386 17631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:02.013504 17631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:59:02.023777 17631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:02.024976 17631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:02.026201 17631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:02.039327 17639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:02.039433 17636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:02.039355 17637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:02.040005 17631 server_base.cc:1034] running on GCE node
I20260929 02:59:02.040771 17631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:02.041740 17631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:02.042966 17631 hybrid_clock.cc:648] HybridClock initialized: now 1790650742042880 us; error 55 us; skew 500 ppm
I20260929 02:59:02.043290 17631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:59:02 dist-test-slave-m1rv krb5kdc[17257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:59:02.046347 17631 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 02:59:02.047717 17631 webserver.cc:460] Webserver started at http://127.8.208.67:34163/ using document root <none> and password file <none>
I20260929 02:59:02.048417 17631 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:02.048514 17631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:02.048755 17631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:02.051573 17631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "14caacf988714d448c956b99d7e256dc"
format_stamp: "Formatted at 2026-09-29 02:59:02 on dist-test-slave-m1rv"
server_key: "cb0b5fe85e25bd4401954114ddffe53d"
server_key_iv: "3c3299eb7869d302628e4fe023f034fb"
server_key_version: "encryptionkey@0"
I20260929 02:59:02.052284 17631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "14caacf988714d448c956b99d7e256dc"
format_stamp: "Formatted at 2026-09-29 02:59:02 on dist-test-slave-m1rv"
server_key: "cb0b5fe85e25bd4401954114ddffe53d"
server_key_iv: "3c3299eb7869d302628e4fe023f034fb"
server_key_version: "encryptionkey@0"
I20260929 02:59:02.057425 17631 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260929 02:59:02.060724 17646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:02.062057 17631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:59:02.062232 17631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "14caacf988714d448c956b99d7e256dc"
format_stamp: "Formatted at 2026-09-29 02:59:02 on dist-test-slave-m1rv"
server_key: "cb0b5fe85e25bd4401954114ddffe53d"
server_key_iv: "3c3299eb7869d302628e4fe023f034fb"
server_key_version: "encryptionkey@0"
I20260929 02:59:02.062412 17631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:02.089449 17631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:02.094122 17631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:02.094436 17631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:02.095427 17631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:02.096787 17631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:02.096863 17631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:02.097034 17631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:02.097086 17631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:02.116492 17491 heartbeater.cc:503] Master 127.8.208.126:37527 was elected leader, sending a full tablet report...
I20260929 02:59:02.130125 17631 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:39725
I20260929 02:59:02.130164 17758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:39725 every 8 connection(s)
I20260929 02:59:02.131342 17631 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:59:02.135352 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 17631
I20260929 02:59:02.135545 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:59:02.136083 9025 external_mini_cluster.cc:1428] Setting key e12175c2740f976e2bbf6b3ef7d5cf17
Sep 29 02:59:02 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:59:02.151752 17627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:02.134069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:37477 (local address 127.8.208.126:37527)
0929 02:59:02.134266 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:02.134272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:02.135595 (+ 1323us) server_negotiation.cc:407] Connection header received
0929 02:59:02.137763 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:02.137767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:02.137844 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:02.137925 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:02.139731 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:02.140473 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:02.142012 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:02.142453 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:02.142561 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:02.145847 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:02.145852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:02.145855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:02.145887 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:02.148661 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:02.149341 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:02.149345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:02.149347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:02.149470 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:02.149911 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:02.149916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:02.149919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:02.150294 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:02.150463 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:02.151191 (+ 728us) server_negotiation.cc:299] Negotiation successful
0929 02:59:02.151422 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:59:02.152680 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:02.134596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:37477)
0929 02:59:02.135408 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:02.135466 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:02.137466 (+ 2000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:02.138146 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:02.138155 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:02.138860 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:02.139579 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:02.139599 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:02.140604 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:02.140608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:02.141882 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:02.141909 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:02.142675 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:02.142680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:02.142856 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:02.143460 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:02.143479 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:02.145669 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:59:02.148784 (+ 3115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:59:02.148790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:59:02.148802 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:59:02.149238 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:59:02.149596 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:59:02.149602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:59:02.149605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:59:02.149789 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:59:02.150423 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:59:02.150432 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:59:02.150795 (+ 363us) client_negotiation.cc:769] Sending connection context
0929 02:59:02.151070 (+ 275us) client_negotiation.cc:240] Negotiation successful
0929 02:59:02.151597 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":297,"threads_started":1}
I20260929 02:59:02.153995 17759 heartbeater.cc:347] Connected to a master server at 127.8.208.126:37527
I20260929 02:59:02.154289 17759 heartbeater.cc:464] Registering TS with master...
I20260929 02:59:02.154979 17759 heartbeater.cc:511] Master 127.8.208.126:37527 requested a full tablet report, sending...
I20260929 02:59:02.156292 17302 ts_manager.cc:194] Registered new tserver with Master: 14caacf988714d448c956b99d7e256dc (127.8.208.67:39725)
I20260929 02:59:02.157078 17302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:37477
I20260929 02:59:02.163534 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab.
Entry for principal oryx/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab.
WARNING: no policy specified for oryx/127.8.208.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab.
Entry for principal oryx/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab.
WARNING: no policy specified for oryx/127.8.208.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab.
Entry for principal oryx/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab.
I20260929 02:59:02.300660 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 17363
Sep 29 02:59:02 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:59:02.316834 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:02.301888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35584 (local address 127.8.208.65:46731)
0929 02:59:02.302553 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:02.302561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:02.302619 (+ 58us) server_negotiation.cc:407] Connection header received
0929 02:59:02.302772 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:02.302780 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:02.302996 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:02.303144 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:02.304429 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:02.305560 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:02.306780 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:02.307333 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:02.307443 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:02.311463 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:02.311477 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:02.311493 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:02.311529 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:02.314454 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:02.315091 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:02.315098 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:02.315100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:02.315215 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:02.315513 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:02.315516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:02.315518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:02.315840 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:02.316060 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:02.316293 (+ 233us) server_negotiation.cc:299] Negotiation successful
0929 02:59:02.316508 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":313,"threads_started":1}
I20260929 02:59:02.318217 17486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:02.401105 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 17363
I20260929 02:59:02.417546 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 17497
Sep 29 02:59:02 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 02:59:02.433167 17771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:02.418372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45570 (local address 127.8.208.66:40847)
0929 02:59:02.418860 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:02.418867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:02.418890 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:59:02.418951 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:02.418957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:02.419156 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:02.419330 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:02.420592 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:02.421664 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:02.423063 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:02.423609 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:02.423758 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:02.427871 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:02.427882 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:02.427896 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:02.427928 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:02.430819 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:02.431450 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:02.431457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:02.431459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:02.431576 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:02.431910 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:02.431913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:02.431915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:02.432215 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:02.432437 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:02.432654 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 02:59:02.432835 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":253,"threads_started":1}
I20260929 02:59:02.434480 17620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:02.517892 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 17497
I20260929 02:59:02.533358 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 17631
Sep 29 02:59:02 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 02:59:02.549963 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:02.534384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46156 (local address 127.8.208.67:39725)
0929 02:59:02.534992 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:02.534999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:02.535026 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:59:02.535083 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:02.535089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:02.535297 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:02.535482 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:02.536813 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:02.537947 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:02.539241 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:02.539805 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:02.539966 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:02.544464 (+ 4498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:02.544476 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:02.544493 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:02.544536 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:02.547292 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:02.547957 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:02.547964 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:02.547966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:02.548092 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:02.548429 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:02.548457 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:02.548462 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:02.548866 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:02.549107 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:02.549478 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 02:59:02.549678 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":325,"threads_started":1}
I20260929 02:59:02.551265 17754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:02.633332 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 17631
I20260929 02:59:02.648818 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 17270
I20260929 02:59:02.650094 17332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:02.754051 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 17270
I20260929 02:59:02.774051 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37527
--webserver_interface=127.8.208.126
--webserver_port=33213
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:37527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:59:03.101336 17776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:03.101629 17776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:03.101984 17776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:03.109793 17776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:59:03.109906 17776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:03.110013 17776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:03.110128 17776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:59:03.110215 17776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:59:03.120297 17776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:37527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=33213
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:03.121680 17776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:03.123056 17776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:03.135524 17784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:03.135604 17781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:03.135592 17782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:03.138690 17776 server_base.cc:1034] running on GCE node
I20260929 02:59:03.139427 17776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:03.140931 17776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:03.142138 17776 hybrid_clock.cc:648] HybridClock initialized: now 1790650743142070 us; error 61 us; skew 500 ppm
I20260929 02:59:03.142515 17776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:59:03 dist-test-slave-m1rv krb5kdc[17257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650743, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:59:03.145777 17776 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:59:03.147348 17776 webserver.cc:460] Webserver started at http://127.8.208.126:33213/ using document root <none> and password file <none>
I20260929 02:59:03.148087 17776 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:03.148245 17776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:03.153687 17776 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:59:03.156422 17791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:03.157740 17776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:59:03.157923 17776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "8e5581e4e3134dae8b28a7a80810fdb8"
format_stamp: "Formatted at 2026-09-29 02:59:00 on dist-test-slave-m1rv"
server_key: "cedee417b4273144680aceb3bf7ae3dd"
server_key_iv: "349ab32eac3c5a963ffb3c547acf747e"
server_key_version: "encryptionkey@0"
I20260929 02:59:03.158907 17776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:03.180342 17776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:03.184404 17776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:03.184686 17776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:03.203357 17776 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:37527
I20260929 02:59:03.203365 17842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:37527 every 8 connection(s)
I20260929 02:59:03.205049 17776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:59:03.205729 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 17776
I20260929 02:59:03.206697 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:46731
--local_ip_for_outbound_sockets=127.8.208.65
--tserver_master_addrs=127.8.208.126:37527
--webserver_port=33743
--webserver_interface=127.8.208.65
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.208.65
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:59:03.212253 17843 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:59:03.217600 17843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8: Bootstrap starting.
I20260929 02:59:03.239044 17843 log.cc:824] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8: Log is configured to *not* fsync() on all Append() calls
I20260929 02:59:03.247838 17843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:59:03.248364 17843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8: Bootstrap complete.
I20260929 02:59:03.257910 17843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37527 } }
I20260929 02:59:03.258764 17843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e5581e4e3134dae8b28a7a80810fdb8, State: Initialized, Role: FOLLOWER
I20260929 02:59:03.259361 17843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37527 } }
I20260929 02:59:03.259614 17843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:59:03.259753 17843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:59:03.259924 17843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:59:03.264065 17843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37527 } }
I20260929 02:59:03.264542 17843 leader_election.cc:302] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e5581e4e3134dae8b28a7a80810fdb8; no voters:
I20260929 02:59:03.265221 17843 leader_election.cc:288] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 02:59:03.265398 17847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 02:59:03.267562 17843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:59:03.272089 17847 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [term 2 LEADER]: Becoming Leader. State: Replica: 8e5581e4e3134dae8b28a7a80810fdb8, State: Running, Role: LEADER
I20260929 02:59:03.272710 17847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37527 } }
I20260929 02:59:03.275712 17848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37527 } } }
I20260929 02:59:03.276618 17848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [sys.catalog]: This master's current role is: LEADER
I20260929 02:59:03.276240 17849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e5581e4e3134dae8b28a7a80810fdb8. Latest consensus state: current_term: 2 leader_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e5581e4e3134dae8b28a7a80810fdb8" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37527 } } }
I20260929 02:59:03.277396 17849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8 [sys.catalog]: This master's current role is: LEADER
I20260929 02:59:03.286319 17859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:59:03.294993 17859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:59:03.296819 17859 catalog_manager.cc:1259] Loaded cluster ID: bcc622fc52b241d581ac5c238937e13a
I20260929 02:59:03.296944 17859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:59:03.302331 17859 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:59:03.304819 17859 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8e5581e4e3134dae8b28a7a80810fdb8: Loaded TSK: 0
I20260929 02:59:03.306778 17859 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 02:59:03.567126 17845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:03.567459 17845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:03.567622 17845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:03.575261 17845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:03.575410 17845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:03.575570 17845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:59:03.585417 17845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.65.keytab
--principal=oryx/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:46731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=33743
--tserver_master_addrs=127.8.208.126:37527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:03.586719 17845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:03.588018 17845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:03.602620 17870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:03.602906 17871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:03.603765 17873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:03.604033 17845 server_base.cc:1034] running on GCE node
I20260929 02:59:03.604698 17845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:03.605832 17845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:03.607056 17845 hybrid_clock.cc:648] HybridClock initialized: now 1790650743607010 us; error 42 us; skew 500 ppm
I20260929 02:59:03.607379 17845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:59:03 dist-test-slave-m1rv krb5kdc[17257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650743, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:59:03.610702 17845 init.cc:377] Logged in from keytab as oryx/127.8.208.65@KRBTEST.COM (short username oryx)
I20260929 02:59:03.612314 17845 webserver.cc:460] Webserver started at http://127.8.208.65:33743/ using document root <none> and password file <none>
I20260929 02:59:03.612998 17845 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:03.613128 17845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:03.618470 17845 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:59:03.621286 17880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:03.622706 17845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:59:03.622879 17845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "0e1b15b9ceb4434b9bd241998f0b0f84"
format_stamp: "Formatted at 2026-09-29 02:59:01 on dist-test-slave-m1rv"
server_key: "c0cb292555e6d7aa79b9eb7689781f2b"
server_key_iv: "097a3b9cae75b7d3f3a69877958443a5"
server_key_version: "encryptionkey@0"
I20260929 02:59:03.623799 17845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:03.647843 17845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:03.652125 17845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:03.652405 17845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:03.653517 17845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:03.654824 17845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:03.654965 17845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:03.655085 17845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:03.655171 17845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:03.690065 17845 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:46731
I20260929 02:59:03.690173 17992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:46731 every 8 connection(s)
I20260929 02:59:03.691480 17845 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:59:03.692106 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 17845
I20260929 02:59:03.692914 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:40847
--local_ip_for_outbound_sockets=127.8.208.66
--tserver_master_addrs=127.8.208.126:37527
--webserver_port=37633
--webserver_interface=127.8.208.66
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.208.66
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:59:03 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:03 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:59:04.023507 17997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:04.023823 17997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:04.023983 17997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:04.031586 17997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:04.031741 17997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:04.031919 17997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:59:04.041646 17997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.66.keytab
--principal=oryx/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:40847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=37633
--tserver_master_addrs=127.8.208.126:37527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:04.043025 17997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:04.044314 17997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:04.059000 18003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:04.058979 18006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:04.059721 18004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:04.061445 17997 server_base.cc:1034] running on GCE node
I20260929 02:59:04.062044 17997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:04.063112 17997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:04.064324 17997 hybrid_clock.cc:648] HybridClock initialized: now 1790650744064258 us; error 64 us; skew 500 ppm
I20260929 02:59:04.064591 17997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:59:04 dist-test-slave-m1rv krb5kdc[17257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650744, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:59:04.068302 17997 init.cc:377] Logged in from keytab as oryx/127.8.208.66@KRBTEST.COM (short username oryx)
I20260929 02:59:04.070076 17997 webserver.cc:460] Webserver started at http://127.8.208.66:37633/ using document root <none> and password file <none>
I20260929 02:59:04.070756 17997 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:04.070910 17997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:04.076388 17997 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260929 02:59:04.079131 18013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:04.080528 17997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:59:04.080683 17997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "3c122f464f414d59aba99c60329ef286"
format_stamp: "Formatted at 2026-09-29 02:59:01 on dist-test-slave-m1rv"
server_key: "145ed309d1519f8e1ecd4b0222dfcba0"
server_key_iv: "ff3dbe0a24b680d1c8cdf9264257a594"
server_key_version: "encryptionkey@0"
I20260929 02:59:04.081678 17997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:04.108557 17997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:04.112939 17997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:04.113299 17997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:04.114569 17997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:04.116011 17997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:04.116097 17997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:04.116243 17997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:04.116317 17997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:04.157387 17997 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:40847
I20260929 02:59:04.157431 18125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:40847 every 8 connection(s)
I20260929 02:59:04.158818 17997 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:59:04.166517 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 17997
I20260929 02:59:04.167547 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:39725
--local_ip_for_outbound_sockets=127.8.208.67
--tserver_master_addrs=127.8.208.126:37527
--webserver_port=34163
--webserver_interface=127.8.208.67
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.208.67
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:59:04 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:04 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:59:04.836663 18130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:04.837054 18130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:04.837246 18130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:04.838696 17996 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.134s user 0.000s sys 0.013s
I20260929 02:59:04.840713 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:03.694881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:33183)
0929 02:59:03.695755 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:03.695798 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:03.697590 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:03.698999 (+ 1409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:03.699009 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:03.699599 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:03.700129 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:03.700145 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:03.701800 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:03.701807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:03.703007 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:03.703018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:03.704055 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:03.704062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:03.704323 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:03.705095 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:03.705122 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:04.839567 (+1134445us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":419,"threads_started":1}
W20260929 02:59:04.841902 17993 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:37527 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.208.126:37527: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
W20260929 02:59:04.842077 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:03.694684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:33183 (local address 127.8.208.126:37527)
0929 02:59:03.695443 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:03.695460 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:03.695944 (+ 484us) server_negotiation.cc:407] Connection header received
0929 02:59:03.698140 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:03.698152 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:03.698476 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:03.698848 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:03.700330 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:03.701646 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:03.703154 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:03.703807 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:03.704016 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:04.840606 (+1136590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:33183: BlockingRecv error: recv got EOF from 127.8.208.65:33183 (error 108)
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":302,"threads_started":1}
W20260929 02:59:04.849128 18130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:04.849294 18130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:04.849511 18130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
Sep 29 02:59:04 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:04 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:59:04.855994 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:04.843143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:58449 (local address 127.8.208.126:37527)
0929 02:59:04.843323 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:04.843329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:04.843868 (+ 539us) server_negotiation.cc:407] Connection header received
0929 02:59:04.844138 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:04.844144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:04.844236 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:04.844343 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:04.845395 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.846497 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.848343 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.848945 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.849050 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:04.855757 (+ 6707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:58449: BlockingRecv error: recv got EOF from 127.8.208.65:58449 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:59:04.856057 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:04.842797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:58449)
0929 02:59:04.843744 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:04.843775 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:04.843981 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:04.844534 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:04.844540 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:04.844991 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:04.845224 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:04.845237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.846658 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.846663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:04.848150 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:04.848163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.849324 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.849332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:04.849531 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:04.850398 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:04.850424 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:04.855563 (+ 5139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":2418,"threads_started":1}
I20260929 02:59:04.864264 18130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/krb5kdc/oryx_127.8.208.67.keytab
--principal=oryx/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:39725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=34163
--tserver_master_addrs=127.8.208.126:37527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
Sep 29 02:59:04 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:04.865914 18130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
Sep 29 02:59:04 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:04.867486 18130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:04.868932 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:04.857089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:39413 (local address 127.8.208.126:37527)
0929 02:59:04.857285 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:04.857291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:04.857434 (+ 143us) server_negotiation.cc:407] Connection header received
0929 02:59:04.857635 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:04.857640 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:04.857715 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:04.857817 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:04.858979 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.860077 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.862008 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.862664 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.862770 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:04.868683 (+ 5913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:39413: BlockingRecv error: recv got EOF from 127.8.208.65:39413 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:59:04.869549 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:04.856986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:39413)
0929 02:59:04.857321 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:04.857348 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:04.857502 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:04.857917 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:04.857924 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:04.858490 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:04.858778 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:04.858794 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.860253 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.860259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:04.861850 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:04.861865 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.862993 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.862999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:04.863161 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:04.864001 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:04.864024 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:04.869292 (+ 5268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197}
W20260929 02:59:04.869853 17993 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 02:59:04.887125 18140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:04.887233 18138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:04.887910 18130 server_base.cc:1034] running on GCE node
W20260929 02:59:04.888737 18137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:04.889650 18130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:04.890712 18130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:04.892145 18130 hybrid_clock.cc:648] HybridClock initialized: now 1790650744892087 us; error 63 us; skew 500 ppm
I20260929 02:59:04.892470 18130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:59:04 dist-test-slave-m1rv krb5kdc[17257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650744, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:59:04.896166 18130 init.cc:377] Logged in from keytab as oryx/127.8.208.67@KRBTEST.COM (short username oryx)
I20260929 02:59:04.897902 18130 webserver.cc:460] Webserver started at http://127.8.208.67:34163/ using document root <none> and password file <none>
I20260929 02:59:04.898583 18130 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:04.898728 18130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:04.904047 18130 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260929 02:59:04.906812 18147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:04.908267 18130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:59:04.908464 18130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "14caacf988714d448c956b99d7e256dc"
format_stamp: "Formatted at 2026-09-29 02:59:02 on dist-test-slave-m1rv"
server_key: "cb0b5fe85e25bd4401954114ddffe53d"
server_key_iv: "3c3299eb7869d302628e4fe023f034fb"
server_key_version: "encryptionkey@0"
I20260929 02:59:04.909453 18130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:04.925495 18130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:04.930056 18130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:04.930331 18130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:04.931399 18130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:04.932832 18130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:04.932914 18130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:04.933059 18130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:04.933141 18130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:04.974705 18130 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:39725
I20260929 02:59:04.974746 18259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:39725 every 8 connection(s)
I20260929 02:59:04.976172 18130 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:59:04.982470 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 18130
Sep 29 02:59:04 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:04 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:05.018826 18263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:04.983514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52716 (local address 127.8.208.126:37527)
0929 02:59:04.993925 (+ 10411us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:04.993934 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:04.993959 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:59:04.994026 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:04.994034 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:04.994131 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:04.994281 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:04.996291 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.998102 (+ 1811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.001523 (+ 3421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.002503 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.002698 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:05.006546 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:05.006565 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:05.006602 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:05.006688 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:05.012740 (+ 6052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:05.014409 (+ 1669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:05.014425 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:05.014440 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:05.014777 (+ 337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:05.015360 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:05.015368 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:05.015374 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:05.016367 (+ 993us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:05.016853 (+ 486us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:05.017552 (+ 699us) server_negotiation.cc:299] Negotiation successful
0929 02:59:05.018153 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10261,"thread_start_us":10124,"threads_started":1}
W20260929 02:59:05.406431 18129 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.234s user 0.004s sys 0.011s
W20260929 02:59:05.408155 18128 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:04.161872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:33735 (local address 127.8.208.126:37527)
0929 02:59:04.162335 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:04.162340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:04.164088 (+ 1748us) server_negotiation.cc:407] Connection header received
0929 02:59:04.166053 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:04.166058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:04.166147 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:04.166272 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:04.167977 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.168920 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.170531 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.170941 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.171087 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:05.407899 (+1236812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:33735: BlockingRecv error: recv got EOF from 127.8.208.66:33735 (error 108)
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":253,"threads_started":1}
I20260929 02:59:05.409971 18129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:04.162957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:33735)
0929 02:59:04.163928 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:04.163966 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:04.165782 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:04.166518 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:04.166526 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:04.167100 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:04.167814 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:04.167838 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.169065 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.169070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:04.170380 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:04.170391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.171170 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.171180 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:04.171413 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:04.172129 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:04.172151 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:05.407802 (+1235651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":360,"threads_started":1}
W20260929 02:59:05.411142 18126 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:37527 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.208.126:37527: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Sep 29 02:59:05 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:05 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:05.423393 18129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:05.412062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:37729)
0929 02:59:05.412458 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:05.412490 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:05.412678 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:05.413026 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:05.413032 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:05.413531 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:05.413772 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:05.413784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.416162 (+ 2378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.416165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:05.417153 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:05.417164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.418754 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.418758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:05.418884 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:05.419562 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:05.419581 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:05.423186 (+ 3605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212}
W20260929 02:59:05.424818 18128 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:05.412035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:37729 (local address 127.8.208.126:37527)
0929 02:59:05.412230 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:05.412238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:05.412569 (+ 331us) server_negotiation.cc:407] Connection header received
0929 02:59:05.412798 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:05.412802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:05.412862 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:05.412978 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:05.413961 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.415978 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.417399 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.418492 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.418698 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:05.423787 (+ 5089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:37729: BlockingRecv error: recv got EOF from 127.8.208.66:37729 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 29 02:59:05 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:05 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:05.437805 18129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:05.424227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:47159)
0929 02:59:05.425233 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:05.425262 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:05.425494 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:05.425914 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:05.425922 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:05.426573 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:05.426919 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:05.426937 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.427999 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.428008 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:05.430525 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:05.430548 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.431423 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.431433 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:05.431736 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:05.432504 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:05.432541 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:05.437523 (+ 4982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":807}
W20260929 02:59:05.438037 18263 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:05.424495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:47159 (local address 127.8.208.126:37527)
0929 02:59:05.424862 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:05.424867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:05.425360 (+ 493us) server_negotiation.cc:407] Connection header received
0929 02:59:05.425641 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:05.425646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:05.425720 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:05.425821 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:05.427110 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.427824 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.430796 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.431229 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.431342 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:05.437750 (+ 6408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:47159: BlockingRecv error: recv got EOF from 127.8.208.66:47159 (error 108)
Metrics: {"server-negotiator.queue_time_us":219}
W20260929 02:59:05.438642 18126 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 02:59:05 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:05 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:05.883519 18266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:05.870613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:37119)
0929 02:59:05.871318 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:05.871341 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:05.871545 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:05.872279 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:05.872286 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:05.873031 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:05.873281 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:05.873296 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.875934 (+ 2638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.875939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:05.877461 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:05.877476 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.878603 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.878612 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:05.878824 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:05.879676 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:05.879701 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:05.883157 (+ 3456us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":1075,"threads_started":1}
W20260929 02:59:05.883605 18263 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:05.870759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:37119 (local address 127.8.208.126:37527)
0929 02:59:05.871104 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:05.871114 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:05.871427 (+ 313us) server_negotiation.cc:407] Connection header received
0929 02:59:05.871804 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:05.871816 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:05.872003 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:05.872161 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:05.873521 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.875444 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.877632 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:05.878327 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:05.878469 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:05.883378 (+ 4909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:37119: BlockingRecv error: recv got EOF from 127.8.208.65:37119 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
W20260929 02:59:06.186436 18262 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.191s user 0.001s sys 0.013s
W20260929 02:59:06.187511 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:04.979141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:38799 (local address 127.8.208.126:37527)
0929 02:59:04.979338 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:04.979343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:04.981236 (+ 1893us) server_negotiation.cc:407] Connection header received
0929 02:59:04.983634 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:04.983643 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:04.983744 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:04.983836 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:04.986853 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.988328 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.991467 (+ 3139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.992396 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.992520 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:06.187311 (+1194791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:38799: BlockingRecv error: recv got EOF from 127.8.208.67:38799 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:59:06.188122 18262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:04.980178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:38799)
0929 02:59:04.981044 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:04.981084 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:04.983306 (+ 2222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:04.984468 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:04.984484 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:04.985518 (+ 1034us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:04.986597 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:04.986627 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.988593 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.988601 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:04.991244 (+ 2643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:04.991263 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:04.992930 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:04.992940 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:04.994504 (+ 1564us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:04.995460 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:04.995494 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:06.187151 (+1191657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":393,"threads_started":1}
W20260929 02:59:06.188998 18260 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:37527 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.208.126:37527: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Sep 29 02:59:06 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:06 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:59:06.205307 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:06.189752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:37685 (local address 127.8.208.126:37527)
0929 02:59:06.189949 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:06.189953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:06.189970 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:59:06.190107 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:06.190110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:06.190166 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:06.190247 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:06.190894 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.191788 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.193277 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.193722 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.193823 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:06.205017 (+ 11194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:37685: BlockingRecv error: recv got EOF from 127.8.208.67:37685 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260929 02:59:06.205734 18262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:06.189695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:37685)
0929 02:59:06.189876 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:06.189899 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:06.190001 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:06.190283 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:06.190286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:06.190618 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:06.190773 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:06.190781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.191931 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.191942 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:06.193128 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:06.193145 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.193873 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.193877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:06.193997 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:06.201033 (+ 7036us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:06.201052 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:06.205352 (+ 4300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
Sep 29 02:59:06 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:06 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:06.215345 18262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:06.206409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:44919)
0929 02:59:06.206684 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:06.206697 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:06.206785 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:06.207089 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:06.207092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:06.207356 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:06.207516 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:06.207524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.208913 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.208917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:06.210071 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:06.210083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.211000 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.211004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:06.211121 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:06.211706 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:06.211722 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:06.215059 (+ 3337us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 02:59:06.215463 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:06.206513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:44919 (local address 127.8.208.126:37527)
0929 02:59:06.206647 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:06.206651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:06.206763 (+ 112us) server_negotiation.cc:407] Connection header received
0929 02:59:06.206879 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:06.206884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:06.206962 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:06.207056 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:06.207655 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.208778 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.210195 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.210854 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.210986 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:06.215250 (+ 4264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:44919: BlockingRecv error: recv got EOF from 127.8.208.67:44919 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260929 02:59:06.215689 18260 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 02:59:06 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:06 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:06.448535 18267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:06.439384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:38457)
0929 02:59:06.439910 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:06.439925 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:06.440083 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:06.440455 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:06.440460 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:06.441018 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:06.441159 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:06.441199 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.442546 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.442550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:06.443508 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:06.443519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.444202 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.444206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:06.444319 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:06.444950 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:06.444967 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:06.448301 (+ 3334us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":302,"threads_started":1}
W20260929 02:59:06.448711 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:06.439483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:38457 (local address 127.8.208.126:37527)
0929 02:59:06.439710 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:06.439718 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:06.440016 (+ 298us) server_negotiation.cc:407] Connection header received
0929 02:59:06.440207 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:06.440213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:06.440300 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:06.440438 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:06.441357 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.442414 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.443645 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.444054 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.444147 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:06.448495 (+ 4348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:38457: BlockingRecv error: recv got EOF from 127.8.208.66:38457 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 29 02:59:06 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:06 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:06.893754 18268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:06.885031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:59149)
0929 02:59:06.885545 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:06.885561 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:06.885712 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:06.886049 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:06.886054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:06.886553 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:06.886731 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:06.886740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.887705 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.887711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:06.888627 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:06.888637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.889489 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.889494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:06.889664 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:06.890253 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:06.890269 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:06.893513 (+ 3244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":304,"threads_started":1}
W20260929 02:59:06.893925 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:06.885090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:59149 (local address 127.8.208.126:37527)
0929 02:59:06.885376 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:06.885384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:06.885643 (+ 259us) server_negotiation.cc:407] Connection header received
0929 02:59:06.885835 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:06.885841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:06.885915 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:06.886017 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:06.886867 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.887575 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.888764 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:06.889327 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:06.889474 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:06.893698 (+ 4224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:59149: BlockingRecv error: recv got EOF from 127.8.208.65:59149 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Sep 29 02:59:07 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:07 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:07.224967 18269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:07.216239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:37073)
0929 02:59:07.216847 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:07.216867 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:07.216978 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:07.217278 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:07.217281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:07.217693 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:07.217830 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:07.217837 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:07.218789 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:07.218793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:07.219739 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:07.219748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:07.220378 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:07.220382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:07.220511 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:07.221221 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:07.221253 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:07.224707 (+ 3454us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":362,"threads_started":1}
W20260929 02:59:07.225068 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:07.216320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:37073 (local address 127.8.208.126:37527)
0929 02:59:07.216503 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:07.216507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:07.216944 (+ 437us) server_negotiation.cc:407] Connection header received
0929 02:59:07.217075 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:07.217079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:07.217138 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:07.217240 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:07.217949 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:07.218670 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:07.219860 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:07.220229 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:07.220321 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:07.224878 (+ 4557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:37073: BlockingRecv error: recv got EOF from 127.8.208.67:37073 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 29 02:59:07 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:07 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:07.458292 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:07.449456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:48035)
0929 02:59:07.449946 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:07.449961 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:07.450115 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:07.450443 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:07.450448 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:07.450902 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:07.451081 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:07.451089 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:07.452059 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:07.452065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:07.453000 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:07.453010 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:07.453766 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:07.453772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:07.453967 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:07.454555 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:07.454574 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:07.457928 (+ 3354us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":269,"threads_started":1}
W20260929 02:59:07.458348 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:07.449524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:48035 (local address 127.8.208.126:37527)
0929 02:59:07.449748 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:07.449757 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:07.450051 (+ 294us) server_negotiation.cc:407] Connection header received
0929 02:59:07.450237 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:07.450244 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:07.450307 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:07.450410 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:07.451224 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:07.451918 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:07.453143 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:07.453602 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:07.453733 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:07.458131 (+ 4398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:48035: BlockingRecv error: recv got EOF from 127.8.208.66:48035 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 29 02:59:07 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:07 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:07.903787 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:07.894692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:54391)
0929 02:59:07.895419 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:07.895448 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:07.895607 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:07.895935 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:07.895940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:07.896329 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:07.896523 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:07.896532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:07.897641 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:07.897647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:07.898572 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:07.898582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:07.899313 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:07.899319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:07.899503 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:07.900057 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:07.900075 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:07.903506 (+ 3431us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":507,"threads_started":1}
W20260929 02:59:07.903924 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:07.894840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:54391 (local address 127.8.208.126:37527)
0929 02:59:07.895072 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:07.895079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:07.895534 (+ 455us) server_negotiation.cc:407] Connection header received
0929 02:59:07.895726 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:07.895732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:07.895805 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:07.895909 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:07.896670 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:07.897508 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:07.898727 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:07.899164 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:07.899301 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:07.903725 (+ 4424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:54391: BlockingRecv error: recv got EOF from 127.8.208.65:54391 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 29 02:59:08 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:08 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:08.234905 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:08.226114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:51541)
0929 02:59:08.226660 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:08.226681 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:08.226795 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:08.227115 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:08.227118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:08.227561 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:08.227742 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:08.227751 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:08.228822 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:08.228826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:08.229832 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:08.229843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:08.230585 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:08.230589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:08.230705 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:08.231329 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:08.231346 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:08.234661 (+ 3315us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":317,"threads_started":1}
W20260929 02:59:08.235080 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:08.226196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:51541 (local address 127.8.208.126:37527)
0929 02:59:08.226408 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:08.226415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:08.226796 (+ 381us) server_negotiation.cc:407] Connection header received
0929 02:59:08.226893 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:08.226897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:08.226962 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:08.227054 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:08.227911 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:08.228691 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:08.229960 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:08.230449 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:08.230547 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:08.234836 (+ 4289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:51541: BlockingRecv error: recv got EOF from 127.8.208.67:51541 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 29 02:59:08 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:08 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:08.468586 18273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:08.459377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:49315)
0929 02:59:08.459882 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:08.459898 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:08.460017 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:08.460306 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:08.460309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:08.460661 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:08.460822 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:08.460830 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:08.461965 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:08.461969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:08.463348 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:08.463369 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:08.464053 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:08.464058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:08.464247 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:08.464867 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:08.464884 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:08.468262 (+ 3378us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":289,"threads_started":1}
W20260929 02:59:08.468743 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:08.459578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:49315 (local address 127.8.208.126:37527)
0929 02:59:08.459802 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:08.459808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:08.459983 (+ 175us) server_negotiation.cc:407] Connection header received
0929 02:59:08.460116 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:08.460119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:08.460178 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:08.460259 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:08.460957 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:08.461777 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:08.463492 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:08.463900 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:08.464031 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:08.468480 (+ 4449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:49315: BlockingRecv error: recv got EOF from 127.8.208.66:49315 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Sep 29 02:59:08 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:08 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:08.914309 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:08.904868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:48871)
0929 02:59:08.905422 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:08.905439 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:08.905601 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:08.905948 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:08.905952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:08.906360 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:08.906527 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:08.906536 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:08.907569 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:08.907575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:08.908619 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:08.908631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:08.909417 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:08.909423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:08.909598 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:08.910162 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:08.910179 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:08.913947 (+ 3768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":302,"threads_started":1}
W20260929 02:59:08.914404 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:08.904932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:48871 (local address 127.8.208.126:37527)
0929 02:59:08.905166 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:08.905195 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:08.905530 (+ 335us) server_negotiation.cc:407] Connection header received
0929 02:59:08.905725 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:08.905731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:08.905806 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:08.905910 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:08.906676 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:08.907418 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:08.908770 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:08.909267 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:08.909395 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:08.914149 (+ 4754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:48871: BlockingRecv error: recv got EOF from 127.8.208.65:48871 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 29 02:59:09 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:09 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:09.244581 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:09.235908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:48789)
0929 02:59:09.236468 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:09.236488 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:09.236595 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:09.236909 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:09.236912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:09.237374 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:09.237545 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:09.237554 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:09.238881 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:09.238885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:09.239810 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:09.239820 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:09.240418 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:09.240422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:09.240559 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:09.241224 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:09.241241 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:09.244336 (+ 3095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":281,"threads_started":1}
W20260929 02:59:09.244715 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:09.236013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:48789 (local address 127.8.208.126:37527)
0929 02:59:09.236177 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:09.236182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:09.236564 (+ 382us) server_negotiation.cc:407] Connection header received
0929 02:59:09.236691 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:09.236695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:09.236771 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:09.236886 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:09.237746 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:09.238749 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:09.239926 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:09.240305 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:09.240393 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:09.244517 (+ 4124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:48789: BlockingRecv error: recv got EOF from 127.8.208.67:48789 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 29 02:59:09 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:09 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:09.478550 18276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:09.469636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:52699)
0929 02:59:09.470103 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:09.470120 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:09.470232 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:09.470603 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:09.470606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:09.470973 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:09.471132 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:09.471140 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:09.472084 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:09.472087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:09.473120 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:09.473131 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:09.474064 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:09.474072 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:09.474255 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:09.474972 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:09.474989 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:09.478282 (+ 3293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":259,"threads_started":1}
W20260929 02:59:09.478725 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:09.469669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:52699 (local address 127.8.208.126:37527)
0929 02:59:09.469880 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:09.469889 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:09.470199 (+ 310us) server_negotiation.cc:407] Connection header received
0929 02:59:09.470365 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:09.470370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:09.470466 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:09.470570 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:09.471251 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:09.471967 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:09.473304 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:09.473888 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:09.474041 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:09.478491 (+ 4450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:52699: BlockingRecv error: recv got EOF from 127.8.208.66:52699 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 29 02:59:09 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:09 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:09.925092 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:09.915767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:33185)
0929 02:59:09.916245 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:09.916261 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:09.916435 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:09.916782 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:09.916787 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:09.917239 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:09.917377 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:09.917386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:09.918478 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:09.918516 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:09.919450 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:09.919460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:09.920151 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:09.920156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:09.920336 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:09.920958 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:09.920975 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:09.924749 (+ 3774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":265,"threads_started":1}
W20260929 02:59:09.925160 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:09.915755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:33185 (local address 127.8.208.126:37527)
0929 02:59:09.915962 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:09.915969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:09.916356 (+ 387us) server_negotiation.cc:407] Connection header received
0929 02:59:09.916568 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:09.916573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:09.916647 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:09.916748 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:09.917530 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:09.918321 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:09.919590 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:09.920006 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:09.920151 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:09.924938 (+ 4787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:33185: BlockingRecv error: recv got EOF from 127.8.208.65:33185 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 29 02:59:10 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:10 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:10.254057 18278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:10.245687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:47281)
0929 02:59:10.246224 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:10.246244 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:10.246353 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:10.246643 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:10.246646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:10.247006 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:10.247161 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:10.247169 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:10.248097 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:10.248101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:10.249084 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:10.249095 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:10.249754 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:10.249758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:10.249873 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:10.250425 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:10.250441 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:10.253738 (+ 3297us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":297,"threads_started":1}
W20260929 02:59:10.254237 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:10.245742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:47281 (local address 127.8.208.126:37527)
0929 02:59:10.245976 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:10.245985 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:10.246326 (+ 341us) server_negotiation.cc:407] Connection header received
0929 02:59:10.246436 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:10.246441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:10.246512 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:10.246591 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:10.247278 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:10.247988 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:10.249217 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:10.249636 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:10.249727 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:10.253982 (+ 4255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:47281: BlockingRecv error: recv got EOF from 127.8.208.67:47281 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 29 02:59:10 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:10 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:10.488277 18279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:10.479457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:51455)
0929 02:59:10.479955 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:10.479970 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:10.480121 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:10.480438 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:10.480443 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:10.480861 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:10.481003 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:10.481011 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:10.482041 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:10.482049 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:10.482962 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:10.482972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:10.483628 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:10.483633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:10.483826 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:10.484442 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:10.484459 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:10.487949 (+ 3490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":303,"threads_started":1}
W20260929 02:59:10.488556 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:10.479651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:51455 (local address 127.8.208.126:37527)
0929 02:59:10.479851 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:10.479858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:10.480050 (+ 192us) server_negotiation.cc:407] Connection header received
0929 02:59:10.480240 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:10.480249 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:10.480306 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:10.480403 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:10.481142 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:10.481916 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:10.483093 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:10.483492 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:10.483618 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:10.488153 (+ 4535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:51455: BlockingRecv error: recv got EOF from 127.8.208.66:51455 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 29 02:59:10 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:10 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:10.935541 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:10.926168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:38555)
0929 02:59:10.926589 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:10.926605 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:10.926719 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:10.927034 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:10.927038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:10.927479 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:10.927667 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:10.927676 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:10.928766 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:10.928772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:10.930336 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:10.930350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:10.931030 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:10.931035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:10.931167 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:10.931795 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:10.931814 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:10.935210 (+ 3396us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":223,"threads_started":1}
W20260929 02:59:10.935650 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:10.926223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:38555 (local address 127.8.208.126:37527)
0929 02:59:10.926459 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:10.926467 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:10.926683 (+ 216us) server_negotiation.cc:407] Connection header received
0929 02:59:10.926809 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:10.926813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:10.926887 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:10.927003 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:10.927809 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:10.928584 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:10.930469 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:10.930875 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:10.930970 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:10.935391 (+ 4421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:38555: BlockingRecv error: recv got EOF from 127.8.208.65:38555 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 29 02:59:11 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:11 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:11.269431 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:11.255200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:50457)
0929 02:59:11.255764 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:11.255785 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:11.255898 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:11.256172 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:11.256175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:11.256661 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:11.256808 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:11.256815 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:11.257802 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:11.257806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:11.258724 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:11.258734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:11.259376 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:11.259380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:11.259512 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:11.265826 (+ 6314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:11.265844 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:11.269164 (+ 3320us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":328,"threads_started":1}
W20260929 02:59:11.269618 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:11.255306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:50457 (local address 127.8.208.126:37527)
0929 02:59:11.255477 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:11.255482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:11.255860 (+ 378us) server_negotiation.cc:407] Connection header received
0929 02:59:11.255990 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:11.255994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:11.256049 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:11.256127 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:11.256933 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:11.257693 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:11.258837 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:11.259251 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:11.259339 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:11.269407 (+ 10068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:50457: BlockingRecv error: recv got EOF from 127.8.208.67:50457 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 02:59:11 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:11 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:11.500135 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:11.489255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:57455)
0929 02:59:11.489735 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:11.489750 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:11.489910 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:11.490273 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:11.490277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:11.490744 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:11.490920 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:11.490929 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:11.492408 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:11.492414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:11.493519 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:11.493531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:11.494448 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:11.494457 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:11.494729 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:11.495548 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:11.495579 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:11.499798 (+ 4219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":253,"threads_started":1}
W20260929 02:59:11.500216 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:11.489304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:57455 (local address 127.8.208.126:37527)
0929 02:59:11.489606 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:11.489612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:11.489834 (+ 222us) server_negotiation.cc:407] Connection header received
0929 02:59:11.490058 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:11.490065 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:11.490147 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:11.490250 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:11.491062 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:11.492244 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:11.493664 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:11.494279 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:11.494415 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:11.500000 (+ 5585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:57455: BlockingRecv error: recv got EOF from 127.8.208.66:57455 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 29 02:59:11 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:11 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:11.945642 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:11.936743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:58355)
0929 02:59:11.937342 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:11.937358 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:11.937485 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:11.937770 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:11.937774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:11.938160 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:11.938320 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:11.938328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:11.939469 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:11.939475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:11.940579 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:11.940595 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:11.941390 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:11.941397 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:11.941573 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:11.942129 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:11.942148 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:11.945392 (+ 3244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":254,"threads_started":1}
W20260929 02:59:11.945817 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:11.936823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:58355 (local address 127.8.208.126:37527)
0929 02:59:11.937049 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:11.937057 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:11.937432 (+ 375us) server_negotiation.cc:407] Connection header received
0929 02:59:11.937582 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:11.937585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:11.937645 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:11.937727 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:11.938434 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:11.939314 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:11.940733 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:11.941200 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:11.941365 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:11.945586 (+ 4221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:58355: BlockingRecv error: recv got EOF from 127.8.208.65:58355 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 29 02:59:12 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:12 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:12.279320 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:12.270362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:49533)
0929 02:59:12.270891 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:12.270925 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:12.271035 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:12.271332 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:12.271336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:12.271838 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:12.272093 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:12.272105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:12.273412 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:12.273417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:12.274350 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:12.274361 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:12.275079 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:12.275085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:12.275259 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:12.275829 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:12.275845 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:12.279100 (+ 3255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":287,"threads_started":1}
W20260929 02:59:12.279551 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:12.270472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:49533 (local address 127.8.208.126:37527)
0929 02:59:12.270636 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:12.270641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:12.271002 (+ 361us) server_negotiation.cc:407] Connection header received
0929 02:59:12.271129 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:12.271132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:12.271190 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:12.271271 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:12.272222 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:12.273271 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:12.274499 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:12.274915 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:12.275058 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:12.279285 (+ 4227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:49533: BlockingRecv error: recv got EOF from 127.8.208.67:49533 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 29 02:59:12 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:12 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:12.510321 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:12.501006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:59551)
0929 02:59:12.501598 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:12.501617 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:12.501783 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:12.502111 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:12.502116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:12.502525 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:12.502687 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:12.502695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:12.503707 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:12.503733 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:12.504661 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:12.504671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:12.505452 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:12.505457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:12.505632 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:12.506267 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:12.506284 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:12.509995 (+ 3711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":304,"threads_started":1}
W20260929 02:59:12.510468 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:12.501097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:59551 (local address 127.8.208.126:37527)
0929 02:59:12.501327 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:12.501336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:12.501727 (+ 391us) server_negotiation.cc:407] Connection header received
0929 02:59:12.501906 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:12.501911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:12.501978 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:12.502079 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:12.502829 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:12.503563 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:12.504794 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:12.505303 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:12.505443 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:12.510221 (+ 4778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:59551: BlockingRecv error: recv got EOF from 127.8.208.66:59551 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 29 02:59:12 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:12 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:12.956017 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:12.946698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:55575)
0929 02:59:12.947270 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:12.947286 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:12.947454 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:12.947792 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:12.947796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:12.948224 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:12.948380 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:12.948389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:12.949694 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:12.949699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:12.950715 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:12.950726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:12.951503 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:12.951510 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:12.951677 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:12.952220 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:12.952237 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:12.955781 (+ 3544us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":303,"threads_started":1}
W20260929 02:59:12.956287 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:12.946773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:55575 (local address 127.8.208.126:37527)
0929 02:59:12.946971 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:12.946978 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:12.947383 (+ 405us) server_negotiation.cc:407] Connection header received
0929 02:59:12.947581 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:12.947586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:12.947660 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:12.947762 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:12.948550 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:12.949551 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:12.950864 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:12.951298 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:12.951444 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:12.956006 (+ 4562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:55575: BlockingRecv error: recv got EOF from 127.8.208.65:55575 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 02:59:13 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:13 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:13.288715 18287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:13.280175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:51781)
0929 02:59:13.280832 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:13.280860 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:13.280976 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:13.281295 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:13.281299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:13.281683 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:13.281849 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:13.281857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:13.282802 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:13.282805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:13.283762 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:13.283773 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:13.284509 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:13.284515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:13.284648 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:13.285247 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:13.285265 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:13.288473 (+ 3208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":310,"threads_started":1}
W20260929 02:59:13.288851 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:13.280349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:51781 (local address 127.8.208.126:37527)
0929 02:59:13.280504 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:13.280508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:13.280944 (+ 436us) server_negotiation.cc:407] Connection header received
0929 02:59:13.281066 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:13.281070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:13.281125 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:13.281273 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:13.281965 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:13.282675 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:13.283874 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:13.284344 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:13.284437 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:13.288645 (+ 4208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:51781: BlockingRecv error: recv got EOF from 127.8.208.67:51781 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 29 02:59:13 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:13 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:13.520352 18288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:13.511291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:60845)
0929 02:59:13.511843 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:13.511859 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:13.511984 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:13.512265 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:13.512268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:13.512722 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:13.512909 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:13.512917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:13.514464 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:13.514469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:13.515483 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:13.515494 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:13.516135 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:13.516139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:13.516249 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:13.516933 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:13.516951 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:13.520123 (+ 3172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":296,"threads_started":1}
W20260929 02:59:13.520546 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:13.511376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:60845 (local address 127.8.208.126:37527)
0929 02:59:13.511583 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:13.511588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:13.511943 (+ 355us) server_negotiation.cc:407] Connection header received
0929 02:59:13.512081 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:13.512085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:13.512141 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:13.512224 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:13.513062 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:13.514286 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:13.515603 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:13.515989 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:13.516084 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:13.520290 (+ 4206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:60845: BlockingRecv error: recv got EOF from 127.8.208.66:60845 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 29 02:59:13 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:13 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:13.965808 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:13.956973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:60807)
0929 02:59:13.957596 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:13.957612 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:13.957719 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:13.958005 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:13.958008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:13.958386 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:13.958536 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:13.958544 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:13.959494 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:13.959497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:13.960406 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:13.960415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:13.961059 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:13.961063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:13.961241 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:13.961811 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:13.961829 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:13.965581 (+ 3752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":422,"threads_started":1}
W20260929 02:59:13.965984 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:13.957003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:60807 (local address 127.8.208.126:37527)
0929 02:59:13.957239 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:13.957248 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:13.957689 (+ 441us) server_negotiation.cc:407] Connection header received
0929 02:59:13.957809 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:13.957813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:13.957884 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:13.957965 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:13.958649 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:13.959368 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:13.960527 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:13.960924 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:13.961050 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:13.965742 (+ 4692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:60807: BlockingRecv error: recv got EOF from 127.8.208.65:60807 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 02:59:14 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:14 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:14.298132 18290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:14.289656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:58577)
0929 02:59:14.290160 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:14.290179 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:14.290319 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:14.290633 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:14.290637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:14.291035 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:14.291191 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:14.291199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:14.292169 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:14.292175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:14.293080 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:14.293090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:14.293841 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:14.293847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:14.294042 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:14.294676 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:14.294693 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:14.297863 (+ 3170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":268,"threads_started":1}
W20260929 02:59:14.298233 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:14.289731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:58577 (local address 127.8.208.126:37527)
0929 02:59:14.289952 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:14.289960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:14.290263 (+ 303us) server_negotiation.cc:407] Connection header received
0929 02:59:14.290441 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:14.290447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:14.290513 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:14.290611 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:14.291329 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:14.292030 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:14.293240 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:14.293671 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:14.293812 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:14.298070 (+ 4258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:58577: BlockingRecv error: recv got EOF from 127.8.208.67:58577 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 02:59:14 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:14 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:14.530130 18291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:14.521583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:52117)
0929 02:59:14.522120 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:14.522136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:14.522244 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:14.522520 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:14.522524 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:14.522919 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:14.523062 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:14.523070 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:14.524005 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:14.524008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:14.524935 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:14.524944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:14.525589 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:14.525593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:14.525715 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:14.526275 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:14.526292 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:14.529906 (+ 3614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":293,"threads_started":1}
W20260929 02:59:14.530356 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:14.521618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:52117 (local address 127.8.208.126:37527)
0929 02:59:14.521811 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:14.521817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:14.522203 (+ 386us) server_negotiation.cc:407] Connection header received
0929 02:59:14.522325 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:14.522328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:14.522387 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:14.522468 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:14.523182 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:14.523880 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:14.525048 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:14.525467 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:14.525555 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:14.530106 (+ 4551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:52117: BlockingRecv error: recv got EOF from 127.8.208.66:52117 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 02:59:14 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:14 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:14.975351 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:14.966679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:48631)
0929 02:59:14.967165 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:14.967180 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:14.967290 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:14.967595 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:14.967598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:14.967997 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:14.968196 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:14.968204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:14.969251 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:14.969257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:14.970436 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:14.970450 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:14.971133 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:14.971138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:14.971248 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:14.971786 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:14.971804 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:14.975014 (+ 3210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":274,"threads_started":1}
W20260929 02:59:14.975374 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:14.966788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:48631 (local address 127.8.208.126:37527)
0929 02:59:14.966947 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:14.966953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:14.967251 (+ 298us) server_negotiation.cc:407] Connection header received
0929 02:59:14.967379 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:14.967382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:14.967439 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:14.967533 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:14.968308 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:14.969075 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:14.970560 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:14.970990 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:14.971085 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:14.975177 (+ 4092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:48631: BlockingRecv error: recv got EOF from 127.8.208.65:48631 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 02:59:15 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:15 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:15.307780 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:15.299120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:37703)
0929 02:59:15.299646 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:15.299666 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:15.299775 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:15.300055 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:15.300058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:15.300430 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:15.300570 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:15.300577 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:15.301623 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:15.301627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:15.302569 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:15.302580 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:15.303218 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:15.303222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:15.303341 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:15.303955 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:15.303972 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:15.307563 (+ 3591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":288,"threads_started":1}
W20260929 02:59:15.307960 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:15.299164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:37703 (local address 127.8.208.126:37527)
0929 02:59:15.299352 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:15.299356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:15.299741 (+ 385us) server_negotiation.cc:407] Connection header received
0929 02:59:15.299868 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:15.299871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:15.299937 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:15.300016 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:15.300687 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:15.301506 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:15.302685 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:15.303097 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:15.303186 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:15.307731 (+ 4545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:37703: BlockingRecv error: recv got EOF from 127.8.208.67:37703 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 02:59:15 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:15 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:15.539919 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:15.531008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:60063)
0929 02:59:15.531483 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:15.531498 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:15.531660 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:15.532023 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:15.532027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:15.532444 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:15.532599 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:15.532608 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:15.533697 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:15.533702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:15.534668 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:15.534678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:15.535390 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:15.535396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:15.535575 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:15.536220 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:15.536237 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:15.539586 (+ 3349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":258,"threads_started":1}
W20260929 02:59:15.540026 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:15.531053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:60063 (local address 127.8.208.126:37527)
0929 02:59:15.531252 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:15.531260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:15.531591 (+ 331us) server_negotiation.cc:407] Connection header received
0929 02:59:15.531797 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:15.531802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:15.531871 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:15.531989 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:15.532750 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:15.533561 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:15.534808 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:15.535230 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:15.535376 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:15.539783 (+ 4407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:60063: BlockingRecv error: recv got EOF from 127.8.208.66:60063 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 02:59:15 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:15 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:15.985492 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:15.976479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:34599)
0929 02:59:15.976968 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:15.976987 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:15.977125 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:15.977544 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:15.977550 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:15.978077 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:15.978316 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:15.978332 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:15.979381 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:15.979388 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:15.980304 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:15.980315 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:15.981063 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:15.981070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:15.981268 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:15.981855 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:15.981873 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:15.985129 (+ 3256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":412,"threads_started":1}
W20260929 02:59:15.985592 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:15.976655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:34599 (local address 127.8.208.126:37527)
0929 02:59:15.976883 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:15.976890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:15.977077 (+ 187us) server_negotiation.cc:407] Connection header received
0929 02:59:15.977291 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:15.977298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:15.977363 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:15.977483 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:15.978490 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:15.979243 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:15.980465 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:15.980896 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:15.981029 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:15.985354 (+ 4325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:34599: BlockingRecv error: recv got EOF from 127.8.208.65:34599 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Sep 29 02:59:16 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:16 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:16.323124 18296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:16.308713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:55931)
0929 02:59:16.309300 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:16.309330 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:16.309439 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:16.309711 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:16.309714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:16.310074 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:16.310226 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:16.310233 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:16.311162 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:16.311165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:16.312222 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:16.312237 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:16.313233 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:16.313239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:16.313417 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:16.319418 (+ 6001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:16.319437 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:16.322773 (+ 3336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":352,"threads_started":1}
W20260929 02:59:16.323179 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:16.308770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:55931 (local address 127.8.208.126:37527)
0929 02:59:16.308952 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:16.308956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:16.309398 (+ 442us) server_negotiation.cc:407] Connection header received
0929 02:59:16.309527 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:16.309530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:16.309587 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:16.309671 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:16.310345 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:16.311045 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:16.312373 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:16.313045 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:16.313225 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:16.322961 (+ 9736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:55931: BlockingRecv error: recv got EOF from 127.8.208.67:55931 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 29 02:59:16 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:16 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:16.549500 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:16.540956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:39583)
0929 02:59:16.541406 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:16.541423 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:16.541544 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:16.541836 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:16.541839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:16.542233 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:16.542402 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:16.542409 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:16.543522 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:16.543526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:16.544405 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:16.544415 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:16.545087 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:16.545091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:16.545250 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:16.545895 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:16.545913 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:16.549276 (+ 3363us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":268,"threads_started":1}
W20260929 02:59:16.549700 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:16.541107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:39583 (local address 127.8.208.126:37527)
0929 02:59:16.541333 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:16.541339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:16.541507 (+ 168us) server_negotiation.cc:407] Connection header received
0929 02:59:16.541639 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:16.541643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:16.541715 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:16.541794 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:16.542539 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:16.543394 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:16.544538 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:16.544963 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:16.545055 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:16.549442 (+ 4387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:39583: BlockingRecv error: recv got EOF from 127.8.208.66:39583 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Sep 29 02:59:16 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:16 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:16.996001 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:16.986596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:36057)
0929 02:59:16.987292 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:16.987311 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:16.987625 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:16.988028 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:16.988033 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:16.988510 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:16.988688 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:16.988698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:16.989916 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:16.989920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:16.991128 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:16.991141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:16.991932 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:16.991936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:16.992072 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:16.992637 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:16.992654 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:16.995750 (+ 3096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":293,"threads_started":1}
W20260929 02:59:16.996170 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:16.986636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:36057 (local address 127.8.208.126:37527)
0929 02:59:16.986857 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:16.986864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:16.987528 (+ 664us) server_negotiation.cc:407] Connection header received
0929 02:59:16.987789 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:16.987795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:16.987874 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:16.987992 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:16.988868 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:16.989743 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:16.991275 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:16.991773 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:16.991872 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:16.995928 (+ 4056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:36057: BlockingRecv error: recv got EOF from 127.8.208.65:36057 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 29 02:59:17 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:17 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:17.332461 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:17.324005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:37793)
0929 02:59:17.324486 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:17.324506 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:17.324609 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:17.324866 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:17.324869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:17.325303 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:17.325509 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:17.325517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:17.326437 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:17.326441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:17.327404 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:17.327422 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:17.328075 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:17.328084 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:17.328249 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:17.328883 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:17.328900 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:17.332250 (+ 3350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":259,"threads_started":1}
W20260929 02:59:17.332646 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:17.324047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:37793 (local address 127.8.208.126:37527)
0929 02:59:17.324215 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:17.324219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:17.324578 (+ 359us) server_negotiation.cc:407] Connection header received
0929 02:59:17.324701 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:17.324704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:17.324775 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:17.324853 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:17.325621 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:17.326317 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:17.327523 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:17.327923 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:17.328024 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:17.332436 (+ 4412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:37793: BlockingRecv error: recv got EOF from 127.8.208.67:37793 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 29 02:59:17 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:17 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:17.558770 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:17.550466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:43599)
0929 02:59:17.550978 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:17.550995 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:17.551108 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:17.551394 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:17.551398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:17.551800 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:17.551971 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:17.551978 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:17.552909 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:17.552912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:17.553911 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:17.553921 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:17.554599 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:17.554603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:17.554725 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:17.555327 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:17.555343 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:17.558535 (+ 3192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":273,"threads_started":1}
W20260929 02:59:17.558936 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:17.550513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:43599 (local address 127.8.208.126:37527)
0929 02:59:17.550727 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:17.550736 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:17.551076 (+ 340us) server_negotiation.cc:407] Connection header received
0929 02:59:17.551203 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:17.551206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:17.551273 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:17.551352 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:17.552103 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:17.552785 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:17.554033 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:17.554452 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:17.554541 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:17.558712 (+ 4171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:43599: BlockingRecv error: recv got EOF from 127.8.208.66:43599 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 29 02:59:17 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:17 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:18.005621 18301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:17.996991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:60877)
0929 02:59:17.997491 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:17.997507 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:17.997666 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:17.998055 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:17.998059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:17.998462 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:17.998612 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:17.998619 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:17.999564 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:17.999567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:18.000487 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:18.000498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:18.001165 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:18.001218 (+ 53us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:18.001412 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:18.002007 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:18.002025 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:18.005393 (+ 3368us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":267,"threads_started":1}
W20260929 02:59:18.005848 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:17.997055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:60877 (local address 127.8.208.126:37527)
0929 02:59:17.997284 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:17.997293 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:17.997598 (+ 305us) server_negotiation.cc:407] Connection header received
0929 02:59:17.997791 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:17.997796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:17.997872 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:17.998000 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:17.998742 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:17.999453 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:18.000612 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:18.001046 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:18.001138 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:18.005585 (+ 4447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:60877: BlockingRecv error: recv got EOF from 127.8.208.65:60877 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 29 02:59:18 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:18 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:18.341903 18302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:18.333424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:51951)
0929 02:59:18.333928 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:18.333948 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:18.334053 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:18.334327 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:18.334330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:18.334829 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:18.335004 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:18.335012 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:18.335967 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:18.335970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:18.336879 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:18.336889 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:18.337665 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:18.337669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:18.337793 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:18.338397 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:18.338417 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:18.341669 (+ 3252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":252,"threads_started":1}
W20260929 02:59:18.342133 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:18.333507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:51951 (local address 127.8.208.126:37527)
0929 02:59:18.333739 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:18.333746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:18.334027 (+ 281us) server_negotiation.cc:407] Connection header received
0929 02:59:18.334148 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:18.334152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:18.334211 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:18.334290 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:18.335125 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:18.335851 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:18.337041 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:18.337505 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:18.337645 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:18.341872 (+ 4227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:51951: BlockingRecv error: recv got EOF from 127.8.208.67:51951 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 29 02:59:18 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:18 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:18.568742 18303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:18.559699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:60327)
0929 02:59:18.560221 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:18.560238 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:18.560347 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:18.560651 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:18.560654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:18.561049 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:18.561270 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:18.561280 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:18.562295 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:18.562299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:18.563305 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:18.563339 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:18.564027 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:18.564031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:18.564153 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:18.564792 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:18.564809 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:18.568335 (+ 3526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":296,"threads_started":1}
W20260929 02:59:18.568794 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:18.559840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:60327 (local address 127.8.208.126:37527)
0929 02:59:18.559994 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:18.559998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:18.560303 (+ 305us) server_negotiation.cc:407] Connection header received
0929 02:59:18.560450 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:18.560454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:18.560523 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:18.560605 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:18.561382 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:18.562144 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:18.563465 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:18.563910 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:18.564009 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:18.568544 (+ 4535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:60327: BlockingRecv error: recv got EOF from 127.8.208.66:60327 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 29 02:59:19 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:19 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.65@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:19.015638 18304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:19.006809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.65:45053)
0929 02:59:19.007286 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:19.007301 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:19.007464 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:19.007778 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:19.007783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:19.008171 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:19.008346 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:19.008354 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.009396 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.009402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:19.010349 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:19.010359 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.011072 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.011077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:19.011256 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:19.011829 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:19.011845 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:19.015387 (+ 3542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":246,"threads_started":1}
W20260929 02:59:19.015820 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:19.006914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:45053 (local address 127.8.208.126:37527)
0929 02:59:19.007157 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:19.007163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:19.007385 (+ 222us) server_negotiation.cc:407] Connection header received
0929 02:59:19.007582 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:19.007592 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:19.007648 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:19.007752 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:19.008490 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.009257 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.010494 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.010924 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.011050 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:19.015587 (+ 4537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.65:45053: BlockingRecv error: recv got EOF from 127.8.208.65:45053 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 29 02:59:19 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:19 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.67@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:19.351564 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:19.343036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.67:42221)
0929 02:59:19.343514 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:19.343546 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:19.343657 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:19.343964 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:19.343967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:19.344350 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:19.344492 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:19.344500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.345590 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.345594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:19.346520 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:19.346531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.347221 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.347225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:19.347354 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:19.347975 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:19.347992 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:19.351327 (+ 3335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":268,"threads_started":1}
W20260929 02:59:19.351818 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:19.343097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:42221 (local address 127.8.208.126:37527)
0929 02:59:19.343385 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:19.343389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:19.343623 (+ 234us) server_negotiation.cc:407] Connection header received
0929 02:59:19.343759 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:19.343764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:19.343831 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:19.343918 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:19.344621 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.345470 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.346687 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.347093 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.347192 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:19.351573 (+ 4381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.67:42221: BlockingRecv error: recv got EOF from 127.8.208.67:42221 (error 108)
Metrics: {"server-negotiator.queue_time_us":176}
Sep 29 02:59:19 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:59:19 dist-test-slave-m1rv krb5kdc[17257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.208.66@KRBTEST.COM for oryx/127.8.208.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:59:19.579054 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:19.569897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37527 (local address 127.8.208.66:43563)
0929 02:59:19.570424 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:19.570440 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:19.570549 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:19.570843 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:19.570846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:19.571193 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:19.571377 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:19.571385 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.572365 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.572369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:19.573459 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:19.573470 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.574276 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.574282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:19.574456 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:19.575113 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:19.575132 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:19.578700 (+ 3568us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.208.126:37527: Server oryx/127.8.208.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":278,"threads_started":1}
W20260929 02:59:19.579192 17995 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:19.569964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:43563 (local address 127.8.208.126:37527)
0929 02:59:19.570147 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:19.570151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:19.570512 (+ 361us) server_negotiation.cc:407] Connection header received
0929 02:59:19.570644 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:19.570647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:19.570723 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:19.570804 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:19.571512 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.572245 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.573581 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.574103 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.574256 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:19.578934 (+ 4678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.208.66:43563: BlockingRecv error: recv got EOF from 127.8.208.66:43563 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:59:19.992515 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 17845
I20260929 02:59:20.004356 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:19.993551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59730 (local address 127.8.208.65:46731)
0929 02:59:19.994220 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:19.994226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:19.994250 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:59:19.994311 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:19.994317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:19.994544 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:19.994703 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:19.996048 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.997091 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.998293 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:19.998822 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:19.998929 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:20.001373 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:20.001382 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:20.001395 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:20.001447 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:20.003851 (+ 2404us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.208.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 02:59:20.004141 (+ 290us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59730: Request ticket server kudu/127.8.208.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":385,"threads_started":1}
W20260929 02:59:20.004523 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 17845: Not authorized: Client connection negotiation failed: client connection to 127.8.208.65:46731: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.208.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 02:59:20.004755 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 17845
I20260929 02:59:20.021333 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 17997
I20260929 02:59:20.034005 18309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:20.022488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.8.208.66:40847)
0929 02:59:20.023156 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:20.023163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:20.023194 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:59:20.023263 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:20.023269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:20.023502 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:20.023667 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:20.025520 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:20.026640 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:20.027756 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:20.028299 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:20.028400 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:20.030857 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:20.030865 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:20.030880 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:20.030927 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:20.033448 (+ 2521us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.208.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 02:59:20.033741 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49598: Request ticket server kudu/127.8.208.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":340,"threads_started":1}
W20260929 02:59:20.034166 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 17997: Not authorized: Client connection negotiation failed: client connection to 127.8.208.66:40847: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.208.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 02:59:20.034233 18309 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49598: Request ticket server kudu/127.8.208.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 02:59:20.034329 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 17997
I20260929 02:59:20.047840 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 18130
I20260929 02:59:20.060708 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:20.048935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56230 (local address 127.8.208.67:39725)
0929 02:59:20.049746 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:20.049753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:20.049780 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:59:20.049845 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:20.049852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:20.050104 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:20.050265 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:20.052130 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:20.053126 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:20.054420 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:20.054974 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:20.055075 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:20.057434 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:20.057442 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:20.057458 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:20.057510 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:20.060145 (+ 2635us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.208.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 02:59:20.060409 (+ 264us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56230: Request ticket server kudu/127.8.208.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":503,"threads_started":1}
W20260929 02:59:20.060752 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 18130: Not authorized: Client connection negotiation failed: client connection to 127.8.208.67:39725: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.208.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 02:59:20.060814 18310 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56230: Request ticket server kudu/127.8.208.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 02:59:20.060953 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 18130
I20260929 02:59:20.077466 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 17776
I20260929 02:59:20.078725 17838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:20.195143 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 17776
2026-09-29T02:59:20Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21466 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T02:59:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:59:20Z Disabled control of system clock
I20260929 02:59:20.270094 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:39189
--webserver_interface=127.8.208.126
--webserver_port=0
--builtin_ntp_servers=127.8.208.84:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:39189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 02:59:20.592554 18315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:20.592837 18315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:20.592913 18315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:20.599968 18315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:59:20.600067 18315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:20.600103 18315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:20.600157 18315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:59:20.600199 18315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:59:20.609242 18315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:39189
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:39189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:20.611218 18315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:20.612983 18315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:20.624696 18320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:20.624773 18321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:20.624716 18323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:20.625452 18315 server_base.cc:1034] running on GCE node
I20260929 02:59:20.626184 18315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:20.627509 18315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:20.628726 18315 hybrid_clock.cc:648] HybridClock initialized: now 1790650760628692 us; error 51 us; skew 500 ppm
I20260929 02:59:20.629091 18315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:59:20.631409 18315 webserver.cc:460] Webserver started at http://127.8.208.126:34085/ using document root <none> and password file <none>
I20260929 02:59:20.632099 18315 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:20.632229 18315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:20.632572 18315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:20.635394 18315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "c2e1ce445a5642bc806c3d470669e540"
format_stamp: "Formatted at 2026-09-29 02:59:20 on dist-test-slave-m1rv"
server_key: "05c53cb54881250bfaf7ae0437930687"
server_key_iv: "3cc233219b97be0c62dd2dc2fcbe79de"
server_key_version: "encryptionkey@0"
I20260929 02:59:20.636145 18315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "c2e1ce445a5642bc806c3d470669e540"
format_stamp: "Formatted at 2026-09-29 02:59:20 on dist-test-slave-m1rv"
server_key: "05c53cb54881250bfaf7ae0437930687"
server_key_iv: "3cc233219b97be0c62dd2dc2fcbe79de"
server_key_version: "encryptionkey@0"
I20260929 02:59:20.641124 18315 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 02:59:20.644368 18329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:20.645696 18315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:59:20.645870 18315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "c2e1ce445a5642bc806c3d470669e540"
format_stamp: "Formatted at 2026-09-29 02:59:20 on dist-test-slave-m1rv"
server_key: "05c53cb54881250bfaf7ae0437930687"
server_key_iv: "3cc233219b97be0c62dd2dc2fcbe79de"
server_key_version: "encryptionkey@0"
I20260929 02:59:20.646080 18315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:20.692909 18315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:20.693884 18315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:20.694171 18315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:20.712352 18315 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:39189
I20260929 02:59:20.712374 18380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:39189 every 8 connection(s)
I20260929 02:59:20.713793 18315 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:59:20.717448 18381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:59:20.720255 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 18315
I20260929 02:59:20.720463 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:59:20.721292 9025 external_mini_cluster.cc:1428] Setting key 2fef169f62ab0f21d0dd842e1db92cad
I20260929 02:59:20.729343 18381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540: Bootstrap starting.
I20260929 02:59:20.732182 18381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540: Neither blocks nor log segments found. Creating new log.
I20260929 02:59:20.733234 18381 log.cc:824] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540: Log is configured to *not* fsync() on all Append() calls
I20260929 02:59:20.734936 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:20.723484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53130 (local address 127.8.208.126:39189)
0929 02:59:20.724404 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:20.724416 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:20.724466 (+ 50us) server_negotiation.cc:407] Connection header received
0929 02:59:20.726156 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:20.726177 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:20.726655 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:20.727071 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:20.727674 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:20.728863 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:20.730331 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:20.731033 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:20.731226 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:20.732392 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:20.732401 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:20.732414 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:20.732941 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:20.733019 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:20.733039 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:20.733174 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:20.733411 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 02:59:20.733701 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":317,"threads_started":1}
I20260929 02:59:20.736544 18381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540: No bootstrap required, opened a new log
I20260929 02:59:20.744071 18381 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e1ce445a5642bc806c3d470669e540" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39189 } }
I20260929 02:59:20.744443 18381 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:59:20.744561 18381 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2e1ce445a5642bc806c3d470669e540, State: Initialized, Role: FOLLOWER
I20260929 02:59:20.745124 18381 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e1ce445a5642bc806c3d470669e540" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39189 } }
I20260929 02:59:20.745431 18381 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:59:20.745570 18381 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:59:20.745756 18381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:59:20.748168 18381 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e1ce445a5642bc806c3d470669e540" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39189 } }
I20260929 02:59:20.748667 18381 leader_election.cc:302] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2e1ce445a5642bc806c3d470669e540; no voters:
I20260929 02:59:20.749325 18381 leader_election.cc:288] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:59:20.749452 18386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:59:20.750301 18386 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [term 1 LEADER]: Becoming Leader. State: Replica: c2e1ce445a5642bc806c3d470669e540, State: Running, Role: LEADER
I20260929 02:59:20.750918 18386 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e1ce445a5642bc806c3d470669e540" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39189 } }
I20260929 02:59:20.751516 18381 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:59:20.753993 18387 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2e1ce445a5642bc806c3d470669e540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e1ce445a5642bc806c3d470669e540" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39189 } } }
I20260929 02:59:20.754434 18387 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [sys.catalog]: This master's current role is: LEADER
I20260929 02:59:20.754019 18388 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2e1ce445a5642bc806c3d470669e540. Latest consensus state: current_term: 1 leader_uuid: "c2e1ce445a5642bc806c3d470669e540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e1ce445a5642bc806c3d470669e540" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39189 } } }
I20260929 02:59:20.754668 18388 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540 [sys.catalog]: This master's current role is: LEADER
I20260929 02:59:20.756362 18394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:59:20.761641 18394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:59:20.771814 18394 catalog_manager.cc:1347] Generated new cluster ID: 7ae5b1752ed8440d96ce74a3e4898879
I20260929 02:59:20.771924 18394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:59:20.782133 18394 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:59:20.783349 18394 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:59:20.796202 18394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2e1ce445a5642bc806c3d470669e540: Generated new TSK 0
I20260929 02:59:20.797062 18394 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:59:20.808527 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39189
--builtin_ntp_servers=127.8.208.84:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 02:59:21.171859 18405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:21.172160 18405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:21.172313 18405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:21.179672 18405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:21.179786 18405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:21.179978 18405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:59:21.189352 18405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:21.190699 18405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:21.192507 18405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:21.205722 18410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:21.205752 18411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:21.205688 18413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:21.206519 18405 server_base.cc:1034] running on GCE node
I20260929 02:59:21.207171 18405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:21.208174 18405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:21.209391 18405 hybrid_clock.cc:648] HybridClock initialized: now 1790650761209346 us; error 65 us; skew 500 ppm
I20260929 02:59:21.209733 18405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:59:21.211995 18405 webserver.cc:460] Webserver started at http://127.8.208.65:34059/ using document root <none> and password file <none>
I20260929 02:59:21.212706 18405 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:21.212864 18405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:21.213238 18405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:21.216032 18405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "467ad96f1e984cc5a5420ea7ee71dd48"
format_stamp: "Formatted at 2026-09-29 02:59:21 on dist-test-slave-m1rv"
server_key: "faa20ccc6d4e857119366b7199286179"
server_key_iv: "129984736901f881850459ea85f2c332"
server_key_version: "encryptionkey@0"
I20260929 02:59:21.216853 18405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "467ad96f1e984cc5a5420ea7ee71dd48"
format_stamp: "Formatted at 2026-09-29 02:59:21 on dist-test-slave-m1rv"
server_key: "faa20ccc6d4e857119366b7199286179"
server_key_iv: "129984736901f881850459ea85f2c332"
server_key_version: "encryptionkey@0"
I20260929 02:59:21.221794 18405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260929 02:59:21.224995 18419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:21.226372 18405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:59:21.226572 18405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "467ad96f1e984cc5a5420ea7ee71dd48"
format_stamp: "Formatted at 2026-09-29 02:59:21 on dist-test-slave-m1rv"
server_key: "faa20ccc6d4e857119366b7199286179"
server_key_iv: "129984736901f881850459ea85f2c332"
server_key_version: "encryptionkey@0"
I20260929 02:59:21.226802 18405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:21.239125 18405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:21.239965 18405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:21.240264 18405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:21.241425 18405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:21.242820 18405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:21.242939 18405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:21.243076 18405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:21.243162 18405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:21.275731 18405 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:32861
I20260929 02:59:21.275789 18531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:32861 every 8 connection(s)
I20260929 02:59:21.277062 18405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:59:21.279008 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 18405
I20260929 02:59:21.279202 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:59:21.279858 9025 external_mini_cluster.cc:1428] Setting key d08826e64764af5b331c415bb3024b53
I20260929 02:59:21.283022 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39189
--builtin_ntp_servers=127.8.208.84:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 02:59:21.290319 18534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:21.280080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:49101 (local address 127.8.208.126:39189)
0929 02:59:21.280587 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:21.280592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:21.282079 (+ 1487us) server_negotiation.cc:407] Connection header received
0929 02:59:21.283879 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:21.283884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:21.283974 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:21.284096 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:21.285125 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:21.285877 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:21.287397 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:21.287783 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:21.287927 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:21.289482 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:21.289487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:21.289491 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:21.289510 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:21.289555 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:21.289568 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:21.289646 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:21.289876 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 02:59:21.289977 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":265,"threads_started":1}
I20260929 02:59:21.291280 18535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:21.280824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39189 (local address 127.8.208.65:49101)
0929 02:59:21.281942 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:21.281981 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:21.283626 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:21.284316 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:21.284323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:21.284432 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:21.284964 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:21.284979 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:21.286044 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:21.286048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:21.287248 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:21.287258 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:21.287994 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:21.288004 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:21.288238 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:21.289058 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:59:21.289063 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:21.289129 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:59:21.289263 (+ 134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:59:21.289672 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:59:21.289679 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:59:21.289684 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:59:21.289867 (+ 183us) client_negotiation.cc:240] Negotiation successful
0929 02:59:21.290239 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":352,"threads_started":1}
I20260929 02:59:21.292825 18532 heartbeater.cc:347] Connected to a master server at 127.8.208.126:39189
I20260929 02:59:21.293201 18532 heartbeater.cc:464] Registering TS with master...
I20260929 02:59:21.294174 18532 heartbeater.cc:511] Master 127.8.208.126:39189 requested a full tablet report, sending...
I20260929 02:59:21.296469 18346 ts_manager.cc:194] Registered new tserver with Master: 467ad96f1e984cc5a5420ea7ee71dd48 (127.8.208.65:32861)
I20260929 02:59:21.297909 18346 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.208.65:49101
W20260929 02:59:21.626191 18536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:21.626456 18536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:21.626585 18536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:21.633996 18536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:21.634090 18536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:21.634253 18536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:59:21.643466 18536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:21.644763 18536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:21.646762 18536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:21.661550 18542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:21.661602 18541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:21.661626 18544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:21.662405 18536 server_base.cc:1034] running on GCE node
I20260929 02:59:21.663044 18536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:21.663970 18536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:21.665272 18536 hybrid_clock.cc:648] HybridClock initialized: now 1790650761665227 us; error 58 us; skew 500 ppm
I20260929 02:59:21.665592 18536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:59:21.667732 18536 webserver.cc:460] Webserver started at http://127.8.208.66:43315/ using document root <none> and password file <none>
I20260929 02:59:21.668417 18536 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:21.668552 18536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:21.668869 18536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:21.671641 18536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "cc562ed1b193434a9b6e4301360549ea"
format_stamp: "Formatted at 2026-09-29 02:59:21 on dist-test-slave-m1rv"
server_key: "3457dfdd6d74b2ea9a12e670f7fd5543"
server_key_iv: "0bf790f6a9904a6c87c3d15604190707"
server_key_version: "encryptionkey@0"
I20260929 02:59:21.672425 18536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "cc562ed1b193434a9b6e4301360549ea"
format_stamp: "Formatted at 2026-09-29 02:59:21 on dist-test-slave-m1rv"
server_key: "3457dfdd6d74b2ea9a12e670f7fd5543"
server_key_iv: "0bf790f6a9904a6c87c3d15604190707"
server_key_version: "encryptionkey@0"
I20260929 02:59:21.677453 18536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260929 02:59:21.680714 18550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:21.682217 18536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:59:21.682435 18536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "cc562ed1b193434a9b6e4301360549ea"
format_stamp: "Formatted at 2026-09-29 02:59:21 on dist-test-slave-m1rv"
server_key: "3457dfdd6d74b2ea9a12e670f7fd5543"
server_key_iv: "0bf790f6a9904a6c87c3d15604190707"
server_key_version: "encryptionkey@0"
I20260929 02:59:21.682658 18536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:21.696741 18536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:21.697620 18536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:21.697921 18536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:21.698808 18536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:21.700186 18536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:21.700316 18536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:21.700457 18536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:21.700546 18536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:21.737074 18536 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:35135
I20260929 02:59:21.737112 18662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:35135 every 8 connection(s)
I20260929 02:59:21.738500 18536 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:59:21.746757 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 18536
I20260929 02:59:21.746932 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:59:21.747457 9025 external_mini_cluster.cc:1428] Setting key 1e7df5f7475e98c0b038cc5addd77f69
I20260929 02:59:21.750229 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39189
--builtin_ntp_servers=127.8.208.84:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 02:59:21.751956 18534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:21.741267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:50953 (local address 127.8.208.126:39189)
0929 02:59:21.741542 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:21.741552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:21.743070 (+ 1518us) server_negotiation.cc:407] Connection header received
0929 02:59:21.744724 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:21.744728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:21.744792 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:21.744883 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:21.746068 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:21.747095 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:21.748845 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:21.749389 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:21.749551 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:21.750978 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:21.750988 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:21.750995 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:21.751020 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:21.751060 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:21.751079 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:21.751188 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:21.751418 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 02:59:21.751574 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:59:21.752948 18665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:21.741830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39189 (local address 127.8.208.66:50953)
0929 02:59:21.742892 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:21.742935 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:21.744459 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:21.745134 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:21.745143 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:21.745311 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:21.745918 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:21.745937 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:21.747230 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:21.747234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:21.748715 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:21.748726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:21.749661 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:21.749670 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:21.749908 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:21.750633 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:59:21.750635 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:21.750672 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:59:21.750782 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:59:21.751193 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:59:21.751200 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:59:21.751205 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:59:21.751357 (+ 152us) client_negotiation.cc:240] Negotiation successful
0929 02:59:21.751765 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":325,"threads_started":1}
I20260929 02:59:21.754282 18663 heartbeater.cc:347] Connected to a master server at 127.8.208.126:39189
I20260929 02:59:21.754592 18663 heartbeater.cc:464] Registering TS with master...
I20260929 02:59:21.755496 18663 heartbeater.cc:511] Master 127.8.208.126:39189 requested a full tablet report, sending...
I20260929 02:59:21.757036 18346 ts_manager.cc:194] Registered new tserver with Master: cc562ed1b193434a9b6e4301360549ea (127.8.208.66:35135)
I20260929 02:59:21.757946 18346 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.208.66:50953
W20260929 02:59:22.078688 18666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:22.078963 18666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:22.079107 18666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:22.086262 18666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:22.086354 18666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:22.086557 18666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:59:22.095542 18666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:22.097288 18666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:22.099013 18666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:22.112326 18672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:22.112375 18674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:22.112377 18671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:22.117522 18666 server_base.cc:1034] running on GCE node
I20260929 02:59:22.118184 18666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:22.119148 18666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:22.120342 18666 hybrid_clock.cc:648] HybridClock initialized: now 1790650762120276 us; error 61 us; skew 500 ppm
I20260929 02:59:22.120661 18666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:59:22.122907 18666 webserver.cc:460] Webserver started at http://127.8.208.67:42917/ using document root <none> and password file <none>
I20260929 02:59:22.123617 18666 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:22.123728 18666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:22.124073 18666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:22.126935 18666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "cac595a7147b40bf94b99ba7ed1f257a"
format_stamp: "Formatted at 2026-09-29 02:59:22 on dist-test-slave-m1rv"
server_key: "459eaa0a7ff7cd49dfd2695471e62c64"
server_key_iv: "b0cd8a76f9826d86e1df51d93bc4dec1"
server_key_version: "encryptionkey@0"
I20260929 02:59:22.127692 18666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "cac595a7147b40bf94b99ba7ed1f257a"
format_stamp: "Formatted at 2026-09-29 02:59:22 on dist-test-slave-m1rv"
server_key: "459eaa0a7ff7cd49dfd2695471e62c64"
server_key_iv: "b0cd8a76f9826d86e1df51d93bc4dec1"
server_key_version: "encryptionkey@0"
I20260929 02:59:22.132635 18666 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:59:22.135907 18680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:22.137266 18666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:59:22.137428 18666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "cac595a7147b40bf94b99ba7ed1f257a"
format_stamp: "Formatted at 2026-09-29 02:59:22 on dist-test-slave-m1rv"
server_key: "459eaa0a7ff7cd49dfd2695471e62c64"
server_key_iv: "b0cd8a76f9826d86e1df51d93bc4dec1"
server_key_version: "encryptionkey@0"
I20260929 02:59:22.137637 18666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:22.154726 18666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:22.155560 18666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:22.155838 18666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:22.156785 18666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:22.158151 18666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:22.158268 18666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:22.158402 18666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:22.158489 18666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:22.190500 18666 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:40989
I20260929 02:59:22.190546 18792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:40989 every 8 connection(s)
I20260929 02:59:22.191787 18666 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:59:22.197255 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 18666
I20260929 02:59:22.197435 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:59:22.198047 9025 external_mini_cluster.cc:1428] Setting key 6fb4802055dde763f5f8437e5bcc064e
I20260929 02:59:22.204613 18534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:22.194338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:46585 (local address 127.8.208.126:39189)
0929 02:59:22.194568 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:22.194575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:22.195883 (+ 1308us) server_negotiation.cc:407] Connection header received
0929 02:59:22.197748 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:22.197754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:22.197844 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:22.197957 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:22.199199 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:22.200265 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:22.202233 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:22.202626 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:22.202717 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:22.203988 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:22.203993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:22.203996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:22.204012 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:22.204038 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:22.204049 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:22.204114 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:22.204316 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 02:59:22.204380 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260929 02:59:22.205647 18795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:22.194870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39189 (local address 127.8.208.67:46585)
0929 02:59:22.195734 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:22.195769 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:22.197438 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:22.198185 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:22.198196 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:22.198291 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:22.199010 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:22.199035 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:22.200415 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:22.200423 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:22.202104 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:22.202121 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:22.202798 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:22.202803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:22.203000 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:22.203715 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:59:22.203717 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:22.203754 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:59:22.203827 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:59:22.204136 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:59:22.204143 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:59:22.204149 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 02:59:22.204290 (+ 141us) client_negotiation.cc:240] Negotiation successful
0929 02:59:22.204573 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":257,"threads_started":1}
I20260929 02:59:22.206818 18793 heartbeater.cc:347] Connected to a master server at 127.8.208.126:39189
I20260929 02:59:22.207079 18793 heartbeater.cc:464] Registering TS with master...
I20260929 02:59:22.207779 18793 heartbeater.cc:511] Master 127.8.208.126:39189 requested a full tablet report, sending...
I20260929 02:59:22.208982 18346 ts_manager.cc:194] Registered new tserver with Master: cac595a7147b40bf94b99ba7ed1f257a (127.8.208.67:40989)
I20260929 02:59:22.209789 18346 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.208.67:46585
I20260929 02:59:22.213212 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 02:59:22.219177 18534 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:22.217507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53136 (local address 127.8.208.126:39189)
0929 02:59:22.217663 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:22.217667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:22.218176 (+ 509us) server_negotiation.cc:407] Connection header received
0929 02:59:22.218363 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:22.218368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:22.218449 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:22.218549 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:22.218943 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53136: BlockingRecv error: recv got EOF from 127.0.0.1:53136 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:59:22.219933 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 18405
I20260929 02:59:22.227530 18802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:22.220901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.8.208.65:32861)
0929 02:59:22.221470 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:22.221477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:22.221499 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:59:22.221550 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:22.221555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:22.221752 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:22.221923 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:22.222308 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:22.223270 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:22.224491 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:22.225011 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:22.225148 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:22.226181 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:22.226189 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:22.226200 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:22.226764 (+ 564us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:22.226806 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:22.226825 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:22.226955 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:22.227069 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 02:59:22.227210 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":309,"threads_started":1}
I20260929 02:59:22.228650 18527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:22.310508 18532 heartbeater.cc:503] Master 127.8.208.126:39189 was elected leader, sending a full tablet report...
I20260929 02:59:22.313336 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 18405
I20260929 02:59:22.329730 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 18536
I20260929 02:59:22.337987 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:22.330694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48650 (local address 127.8.208.66:35135)
0929 02:59:22.331246 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:22.331253 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:22.331280 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:59:22.331348 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:22.331354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:22.331579 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:22.331726 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:22.332142 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:22.333137 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:22.334391 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:22.335070 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:22.335215 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:22.336394 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:22.336402 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:22.336418 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:22.337012 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:22.337064 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:22.337083 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:22.337321 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:22.337487 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 02:59:22.337639 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":293,"threads_started":1}
I20260929 02:59:22.339205 18658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:22.420681 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 18536
I20260929 02:59:22.437230 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 18666
I20260929 02:59:22.445714 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:22.438198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34316 (local address 127.8.208.67:40989)
0929 02:59:22.438827 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:22.438833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:22.438861 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:59:22.438918 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:22.438924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:22.439138 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:22.439315 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:22.439780 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:22.440787 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:22.442320 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:22.442880 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:22.443024 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:22.444231 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:22.444240 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:22.444254 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:22.444843 (+ 589us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:22.444891 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:22.444910 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:22.445040 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:22.445227 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 02:59:22.445373 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":347,"threads_started":1}
I20260929 02:59:22.446969 18788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:22.527329 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 18666
I20260929 02:59:22.542501 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 18315
I20260929 02:59:22.543787 18376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:22.643139 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 18315
2026-09-29T02:59:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2441 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T02:59:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:59:22Z Disabled control of system clock
I20260929 02:59:22.711022 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:39627
--webserver_interface=127.8.208.126
--webserver_port=0
--builtin_ntp_servers=127.8.208.84:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:39627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 02:59:23.031036 18812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:23.031301 18812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:23.031467 18812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:23.039228 18812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:59:23.039354 18812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:23.039444 18812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:23.039559 18812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:59:23.039625 18812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:59:23.048887 18812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:39627
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:39627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:23.050140 18812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:23.051867 18812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:23.063381 18817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:23.063468 18820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:23.063439 18818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:23.064356 18812 server_base.cc:1034] running on GCE node
I20260929 02:59:23.065151 18812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:23.066615 18812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:23.067850 18812 hybrid_clock.cc:648] HybridClock initialized: now 1790650763067794 us; error 55 us; skew 500 ppm
I20260929 02:59:23.068203 18812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:59:23.070487 18812 webserver.cc:460] Webserver started at http://127.8.208.126:46005/ using document root <none> and password file <none>
I20260929 02:59:23.071188 18812 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:23.071321 18812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:23.071640 18812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:23.074491 18812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "55aefa76a0294020812c8a7aedf150d9"
format_stamp: "Formatted at 2026-09-29 02:59:23 on dist-test-slave-m1rv"
server_key: "a2813414f6a6280ce9d62d780426c044"
server_key_iv: "0eb9067af97facee8e21211cb1ecbcbe"
server_key_version: "encryptionkey@0"
I20260929 02:59:23.075277 18812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "55aefa76a0294020812c8a7aedf150d9"
format_stamp: "Formatted at 2026-09-29 02:59:23 on dist-test-slave-m1rv"
server_key: "a2813414f6a6280ce9d62d780426c044"
server_key_iv: "0eb9067af97facee8e21211cb1ecbcbe"
server_key_version: "encryptionkey@0"
I20260929 02:59:23.080114 18812 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260929 02:59:23.083439 18826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:23.084631 18812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:59:23.084810 18812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "55aefa76a0294020812c8a7aedf150d9"
format_stamp: "Formatted at 2026-09-29 02:59:23 on dist-test-slave-m1rv"
server_key: "a2813414f6a6280ce9d62d780426c044"
server_key_iv: "0eb9067af97facee8e21211cb1ecbcbe"
server_key_version: "encryptionkey@0"
I20260929 02:59:23.085026 18812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:23.090557 18812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:23.091300 18812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:23.091594 18812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:23.109135 18812 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:39627
I20260929 02:59:23.109233 18877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:39627 every 8 connection(s)
I20260929 02:59:23.110522 18812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:59:23.114174 18878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:59:23.119580 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 18812
I20260929 02:59:23.119827 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:59:23.120437 9025 external_mini_cluster.cc:1428] Setting key 88ab1e3edc8c0226c3fc07522e0cea6e
I20260929 02:59:23.125348 18878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9: Bootstrap starting.
I20260929 02:59:23.128341 18878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9: Neither blocks nor log segments found. Creating new log.
I20260929 02:59:23.129026 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:23.122575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56270 (local address 127.8.208.126:39627)
0929 02:59:23.123612 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:23.123623 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:23.123665 (+ 42us) server_negotiation.cc:407] Connection header received
0929 02:59:23.125365 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:23.125391 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:23.125845 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:23.126220 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:23.127343 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:23.127348 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:23.127361 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:23.127380 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:23.127432 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:23.127448 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:23.127681 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:23.127811 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 02:59:23.128095 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":345,"threads_started":1}
I20260929 02:59:23.129563 18878 log.cc:824] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9: Log is configured to *not* fsync() on all Append() calls
I20260929 02:59:23.132509 18878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9: No bootstrap required, opened a new log
I20260929 02:59:23.139521 18878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55aefa76a0294020812c8a7aedf150d9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39627 } }
I20260929 02:59:23.139829 18878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:59:23.139899 18878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55aefa76a0294020812c8a7aedf150d9, State: Initialized, Role: FOLLOWER
I20260929 02:59:23.140439 18878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55aefa76a0294020812c8a7aedf150d9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39627 } }
I20260929 02:59:23.140671 18878 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:59:23.140818 18878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:59:23.141068 18878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:59:23.143173 18878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55aefa76a0294020812c8a7aedf150d9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39627 } }
I20260929 02:59:23.143659 18878 leader_election.cc:302] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55aefa76a0294020812c8a7aedf150d9; no voters:
I20260929 02:59:23.144254 18878 leader_election.cc:288] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:59:23.144332 18883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:59:23.145207 18883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [term 1 LEADER]: Becoming Leader. State: Replica: 55aefa76a0294020812c8a7aedf150d9, State: Running, Role: LEADER
I20260929 02:59:23.145861 18883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55aefa76a0294020812c8a7aedf150d9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39627 } }
I20260929 02:59:23.146452 18878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:59:23.148645 18884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55aefa76a0294020812c8a7aedf150d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55aefa76a0294020812c8a7aedf150d9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39627 } } }
I20260929 02:59:23.148751 18885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55aefa76a0294020812c8a7aedf150d9. Latest consensus state: current_term: 1 leader_uuid: "55aefa76a0294020812c8a7aedf150d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55aefa76a0294020812c8a7aedf150d9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39627 } } }
I20260929 02:59:23.149040 18884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [sys.catalog]: This master's current role is: LEADER
I20260929 02:59:23.149060 18885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9 [sys.catalog]: This master's current role is: LEADER
I20260929 02:59:23.150667 18891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:59:23.155843 18891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:59:23.164649 18891 catalog_manager.cc:1347] Generated new cluster ID: c7cbd61c88d24a58814d6771c5351910
I20260929 02:59:23.164767 18891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:59:23.182068 18891 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:59:23.182487 18891 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:59:23.189100 18891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55aefa76a0294020812c8a7aedf150d9: Generated new TSK 0
I20260929 02:59:23.190064 18891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:59:23.195449 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39627
--builtin_ntp_servers=127.8.208.84:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 02:59:23.520051 18902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:23.520468 18902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:23.520632 18902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:23.527850 18902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:23.527983 18902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:23.528164 18902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:59:23.537154 18902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:23.538532 18902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:23.540462 18902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:23.554195 18907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:23.554136 18910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:23.554231 18908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:23.555433 18902 server_base.cc:1034] running on GCE node
I20260929 02:59:23.556015 18902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:23.557046 18902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:23.558262 18902 hybrid_clock.cc:648] HybridClock initialized: now 1790650763558222 us; error 61 us; skew 500 ppm
I20260929 02:59:23.558578 18902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:59:23.560701 18902 webserver.cc:460] Webserver started at http://127.8.208.65:36117/ using document root <none> and password file <none>
I20260929 02:59:23.561471 18902 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:23.561594 18902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:23.561913 18902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:23.564658 18902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "503cccb790a545668348965119fd631b"
format_stamp: "Formatted at 2026-09-29 02:59:23 on dist-test-slave-m1rv"
server_key: "f8e8f9ea41e153b359d8f9fb2ca2ca01"
server_key_iv: "37ba7d8906e58900f927b57d520d1820"
server_key_version: "encryptionkey@0"
I20260929 02:59:23.565564 18902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "503cccb790a545668348965119fd631b"
format_stamp: "Formatted at 2026-09-29 02:59:23 on dist-test-slave-m1rv"
server_key: "f8e8f9ea41e153b359d8f9fb2ca2ca01"
server_key_iv: "37ba7d8906e58900f927b57d520d1820"
server_key_version: "encryptionkey@0"
I20260929 02:59:23.570662 18902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:59:23.574057 18916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:23.575301 18902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:59:23.575495 18902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "503cccb790a545668348965119fd631b"
format_stamp: "Formatted at 2026-09-29 02:59:23 on dist-test-slave-m1rv"
server_key: "f8e8f9ea41e153b359d8f9fb2ca2ca01"
server_key_iv: "37ba7d8906e58900f927b57d520d1820"
server_key_version: "encryptionkey@0"
I20260929 02:59:23.575711 18902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:23.581300 18902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:23.582147 18902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:23.582409 18902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:23.583328 18902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:23.584618 18902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:23.584741 18902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:23.584859 18902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:23.584947 18902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:23.617446 18902 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:41387
I20260929 02:59:23.617489 19028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:41387 every 8 connection(s)
I20260929 02:59:23.618775 18902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:59:23.626727 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:23.621328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:43931 (local address 127.8.208.126:39627)
0929 02:59:23.621501 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:23.621505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:23.622818 (+ 1313us) server_negotiation.cc:407] Connection header received
0929 02:59:23.624484 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:23.624490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:23.624558 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:23.624704 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:23.626010 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:23.626016 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:23.626020 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:23.626040 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:23.626079 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:23.626090 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:23.626219 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:23.626392 (+ 173us) server_negotiation.cc:299] Negotiation successful
0929 02:59:23.626468 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:59:23.627624 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:23.621797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39627 (local address 127.8.208.65:43931)
0929 02:59:23.622666 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:23.622701 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:23.624210 (+ 1509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:23.624939 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:23.624954 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:23.625040 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:23.625708 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:59:23.625710 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:23.625746 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:59:23.625839 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:59:23.626181 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:59:23.626191 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:59:23.626201 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 02:59:23.626470 (+ 269us) client_negotiation.cc:240] Negotiation successful
0929 02:59:23.626703 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":252,"threads_started":1}
I20260929 02:59:23.629073 19029 heartbeater.cc:347] Connected to a master server at 127.8.208.126:39627
I20260929 02:59:23.629249 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 18902
I20260929 02:59:23.629426 19029 heartbeater.cc:464] Registering TS with master...
I20260929 02:59:23.629518 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:59:23.630124 19029 heartbeater.cc:511] Master 127.8.208.126:39627 requested a full tablet report, sending...
I20260929 02:59:23.630327 9025 external_mini_cluster.cc:1428] Setting key d2c2d3c06bcb799973f2d3d10688e02b
I20260929 02:59:23.632325 18843 ts_manager.cc:194] Registered new tserver with Master: 503cccb790a545668348965119fd631b (127.8.208.65:41387)
I20260929 02:59:23.633491 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39627
--builtin_ntp_servers=127.8.208.84:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 02:59:23.967252 19032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:23.967583 19032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:23.967754 19032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:23.974905 19032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:23.975046 19032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:23.975222 19032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:59:23.984345 19032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:23.985666 19032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:23.987378 19032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:24.000240 19037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:24.000200 19038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:24.000200 19040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:24.001411 19032 server_base.cc:1034] running on GCE node
I20260929 02:59:24.002158 19032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:24.003135 19032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:24.004345 19032 hybrid_clock.cc:648] HybridClock initialized: now 1790650764004300 us; error 33 us; skew 500 ppm
I20260929 02:59:24.004671 19032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:59:24.007185 19032 webserver.cc:460] Webserver started at http://127.8.208.66:42297/ using document root <none> and password file <none>
I20260929 02:59:24.007891 19032 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:24.008008 19032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:24.008342 19032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:24.011296 19032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "fd5fd4c8fda54397b4276c58c5975f76"
format_stamp: "Formatted at 2026-09-29 02:59:24 on dist-test-slave-m1rv"
server_key: "d941e12cfdc543c04da79d4a8606f998"
server_key_iv: "b77067570c13c7989fe9e878246cbf84"
server_key_version: "encryptionkey@0"
I20260929 02:59:24.012089 19032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "fd5fd4c8fda54397b4276c58c5975f76"
format_stamp: "Formatted at 2026-09-29 02:59:24 on dist-test-slave-m1rv"
server_key: "d941e12cfdc543c04da79d4a8606f998"
server_key_iv: "b77067570c13c7989fe9e878246cbf84"
server_key_version: "encryptionkey@0"
I20260929 02:59:24.017264 19032 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 02:59:24.020658 19046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:24.022141 19032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 02:59:24.022318 19032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "fd5fd4c8fda54397b4276c58c5975f76"
format_stamp: "Formatted at 2026-09-29 02:59:24 on dist-test-slave-m1rv"
server_key: "d941e12cfdc543c04da79d4a8606f998"
server_key_iv: "b77067570c13c7989fe9e878246cbf84"
server_key_version: "encryptionkey@0"
I20260929 02:59:24.022543 19032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:24.028290 19032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:24.029098 19032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:24.029410 19032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:24.030313 19032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:24.031641 19032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:24.031764 19032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:24.031898 19032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:24.031991 19032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:24.064538 19032 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:33959
I20260929 02:59:24.064625 19158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:33959 every 8 connection(s)
I20260929 02:59:24.065908 19032 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:59:24.073676 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 19032
I20260929 02:59:24.073858 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:59:24.074286 9025 external_mini_cluster.cc:1428] Setting key f36bcb06d7ef69ea678db760ac2cd3b2
I20260929 02:59:24.074357 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:24.068544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:41245 (local address 127.8.208.126:39627)
0929 02:59:24.068769 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:24.068775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:24.070230 (+ 1455us) server_negotiation.cc:407] Connection header received
0929 02:59:24.071824 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:24.071829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:24.071882 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:24.071972 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:24.073326 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:24.073332 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:24.073335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:24.073358 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:24.073395 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:24.073411 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:24.073601 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:24.073907 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 02:59:24.073989 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 02:59:24.075441 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:24.069114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39627 (local address 127.8.208.66:41245)
0929 02:59:24.070055 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:24.070090 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:24.071570 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:24.072178 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:24.072186 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:24.072278 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:24.072909 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:59:24.072911 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:24.072956 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:59:24.073037 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:59:24.073559 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:59:24.073576 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:59:24.073586 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 02:59:24.074001 (+ 415us) client_negotiation.cc:240] Negotiation successful
0929 02:59:24.074261 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":261,"threads_started":1}
I20260929 02:59:24.076916 19159 heartbeater.cc:347] Connected to a master server at 127.8.208.126:39627
I20260929 02:59:24.077276 19159 heartbeater.cc:464] Registering TS with master...
I20260929 02:59:24.077517 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39627
--builtin_ntp_servers=127.8.208.84:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 02:59:24.077919 19159 heartbeater.cc:511] Master 127.8.208.126:39627 requested a full tablet report, sending...
I20260929 02:59:24.079298 18843 ts_manager.cc:194] Registered new tserver with Master: fd5fd4c8fda54397b4276c58c5975f76 (127.8.208.66:33959)
W20260929 02:59:24.400701 19162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:24.400986 19162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:24.401077 19162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:24.408371 19162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:24.408470 19162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:24.408577 19162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:59:24.417435 19162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:24.418702 19162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:24.420437 19162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:24.433513 19167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:24.433712 19170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:24.433574 19168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:24.434320 19162 server_base.cc:1034] running on GCE node
I20260929 02:59:24.434896 19162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:24.435887 19162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:24.437098 19162 hybrid_clock.cc:648] HybridClock initialized: now 1790650764437037 us; error 61 us; skew 500 ppm
I20260929 02:59:24.437456 19162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:59:24.439680 19162 webserver.cc:460] Webserver started at http://127.8.208.67:43917/ using document root <none> and password file <none>
I20260929 02:59:24.440353 19162 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:24.440475 19162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:24.440802 19162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:24.443599 19162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "5a0d06ee1c6c4f0c96a62ee5253ddf0d"
format_stamp: "Formatted at 2026-09-29 02:59:24 on dist-test-slave-m1rv"
server_key: "b44b111f2580067a1e4bd1577444bf8c"
server_key_iv: "1f21403004b38739091949f47526e810"
server_key_version: "encryptionkey@0"
I20260929 02:59:24.444331 19162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "5a0d06ee1c6c4f0c96a62ee5253ddf0d"
format_stamp: "Formatted at 2026-09-29 02:59:24 on dist-test-slave-m1rv"
server_key: "b44b111f2580067a1e4bd1577444bf8c"
server_key_iv: "1f21403004b38739091949f47526e810"
server_key_version: "encryptionkey@0"
I20260929 02:59:24.449920 19162 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260929 02:59:24.453493 19176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:24.454721 19162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:59:24.454891 19162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "5a0d06ee1c6c4f0c96a62ee5253ddf0d"
format_stamp: "Formatted at 2026-09-29 02:59:24 on dist-test-slave-m1rv"
server_key: "b44b111f2580067a1e4bd1577444bf8c"
server_key_iv: "1f21403004b38739091949f47526e810"
server_key_version: "encryptionkey@0"
I20260929 02:59:24.455106 19162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:24.460752 19162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:24.461577 19162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:24.461833 19162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:24.462702 19162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:24.464025 19162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:24.464151 19162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:24.464274 19162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:24.464363 19162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:24.496726 19162 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:46093
I20260929 02:59:24.496776 19288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:46093 every 8 connection(s)
I20260929 02:59:24.498078 19162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:59:24.499341 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 19162
I20260929 02:59:24.499564 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:59:24.500103 9025 external_mini_cluster.cc:1428] Setting key 9e613b350faa2c503461fb7d5e6e95a6
I20260929 02:59:24.506628 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:24.500784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:45623 (local address 127.8.208.126:39627)
0929 02:59:24.501007 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:24.501013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:24.502648 (+ 1635us) server_negotiation.cc:407] Connection header received
0929 02:59:24.504463 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:24.504468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:24.504532 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:24.504645 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:24.505918 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:24.505924 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:24.505928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:24.505957 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:24.505991 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:24.506002 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:24.506118 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:24.506305 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 02:59:24.506372 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:59:24.507520 19291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:24.501484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39627 (local address 127.8.208.67:45623)
0929 02:59:24.502503 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:24.502538 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:24.504211 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:24.504875 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:24.504883 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:24.504982 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:24.505640 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:59:24.505643 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:24.505679 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:59:24.505758 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:59:24.506095 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:59:24.506105 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:59:24.506112 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 02:59:24.506386 (+ 274us) client_negotiation.cc:240] Negotiation successful
0929 02:59:24.506639 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":292,"threads_started":1}
I20260929 02:59:24.508687 19289 heartbeater.cc:347] Connected to a master server at 127.8.208.126:39627
I20260929 02:59:24.508936 19289 heartbeater.cc:464] Registering TS with master...
I20260929 02:59:24.509497 19289 heartbeater.cc:511] Master 127.8.208.126:39627 requested a full tablet report, sending...
I20260929 02:59:24.510736 18843 ts_manager.cc:194] Registered new tserver with Master: 5a0d06ee1c6c4f0c96a62ee5253ddf0d (127.8.208.67:46093)
I20260929 02:59:24.515154 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 02:59:24.520816 18881 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:59:24.519338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56276 (local address 127.8.208.126:39627)
0929 02:59:24.519506 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:24.519510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:24.519962 (+ 452us) server_negotiation.cc:407] Connection header received
0929 02:59:24.520097 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:24.520105 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:24.520169 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:24.520263 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:24.520561 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56276: BlockingRecv error: recv got EOF from 127.0.0.1:56276 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:59:24.521426 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 18902
I20260929 02:59:24.524719 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:24.522375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.8.208.65:41387)
0929 02:59:24.522912 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:24.522920 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:24.522943 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:59:24.522997 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:24.523002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:24.523188 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:24.523303 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:24.524169 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:24.524175 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:24.524186 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:24.524204 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:24.524245 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:24.524258 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:24.524357 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:24.524443 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 02:59:24.524513 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":292,"threads_started":1}
I20260929 02:59:24.525991 19024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:24.606662 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 18902
I20260929 02:59:24.622841 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 19032
I20260929 02:59:24.626478 19300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:24.623686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46570 (local address 127.8.208.66:33959)
0929 02:59:24.624160 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:24.624167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:24.624192 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:59:24.624252 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:24.624258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:24.624473 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:24.624643 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:24.625708 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:24.625714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:24.625767 (+ 53us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:24.625791 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:24.625849 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:24.625865 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:24.626017 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:24.626109 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 02:59:24.626205 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":216,"threads_started":1}
I20260929 02:59:24.627785 19154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:24.709255 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 19032
I20260929 02:59:24.726392 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 19162
I20260929 02:59:24.730469 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:24.727435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36506 (local address 127.8.208.67:46093)
0929 02:59:24.728041 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:24.728054 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:24.728085 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:59:24.728143 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:24.728150 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:24.728362 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:24.728545 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:24.729713 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:24.729719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:24.729737 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:59:24.729752 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:24.729790 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:59:24.729803 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:24.730000 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:24.730074 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 02:59:24.730175 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":312,"threads_started":1}
I20260929 02:59:24.731648 19284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:24.815678 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 19162
I20260929 02:59:24.830996 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 18812
I20260929 02:59:24.832110 18873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:24.932202 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 18812
2026-09-29T02:59:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2289 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:59:25 dist-test-slave-m1rv krb5kdc[19307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:59:25 dist-test-slave-m1rv krb5kdc[19307](info): set up 2 sockets
Sep 29 02:59:25 dist-test-slave-m1rv krb5kdc[19307](info): commencing operation
krb5kdc: starting...
W20260929 02:59:26.030131 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:59:26 dist-test-slave-m1rv krb5kdc[19307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:59:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:59:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 02:59:26.253238 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37039
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:37039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:59:26.590271 19320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:26.590643 19320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:26.590828 19320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:26.598204 19320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:59:26.598371 19320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:26.598489 19320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:26.598605 19320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:59:26.598690 19320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:59:26.608971 19320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:37039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:26.610414 19320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:26.611812 19320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:26.623980 19325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:26.624130 19326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:26.624087 19328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:26.625334 19320 server_base.cc:1034] running on GCE node
I20260929 02:59:26.626103 19320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:26.627454 19320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:26.628675 19320 hybrid_clock.cc:648] HybridClock initialized: now 1790650766628619 us; error 43 us; skew 500 ppm
I20260929 02:59:26.629060 19320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:59:26 dist-test-slave-m1rv krb5kdc[19307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650766, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:59:26.638876 19320 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 02:59:26.640301 19320 webserver.cc:460] Webserver started at http://127.8.208.126:38653/ using document root <none> and password file <none>
I20260929 02:59:26.641012 19320 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:26.641163 19320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:26.641531 19320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:26.644464 19320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "68e7892cd5ad442aa10d9b2d5de36ee2"
format_stamp: "Formatted at 2026-09-29 02:59:26 on dist-test-slave-m1rv"
server_key: "e99f78b3c21e244a40b7ff5abc76b4ac"
server_key_iv: "29a6e8bad65d04591b90e2efd1122af5"
server_key_version: "encryptionkey@0"
I20260929 02:59:26.645291 19320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "68e7892cd5ad442aa10d9b2d5de36ee2"
format_stamp: "Formatted at 2026-09-29 02:59:26 on dist-test-slave-m1rv"
server_key: "e99f78b3c21e244a40b7ff5abc76b4ac"
server_key_iv: "29a6e8bad65d04591b90e2efd1122af5"
server_key_version: "encryptionkey@0"
I20260929 02:59:26.650262 19320 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 02:59:26.653750 19335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:26.655336 19320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:59:26.655545 19320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "68e7892cd5ad442aa10d9b2d5de36ee2"
format_stamp: "Formatted at 2026-09-29 02:59:26 on dist-test-slave-m1rv"
server_key: "e99f78b3c21e244a40b7ff5abc76b4ac"
server_key_iv: "29a6e8bad65d04591b90e2efd1122af5"
server_key_version: "encryptionkey@0"
I20260929 02:59:26.655747 19320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:26.677013 19320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:26.680828 19320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:26.681071 19320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:26.699200 19320 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:37039
I20260929 02:59:26.699234 19386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:37039 every 8 connection(s)
I20260929 02:59:26.700568 19320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 02:59:26.703825 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 19320
I20260929 02:59:26.704062 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 02:59:26.704576 19387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:59:26.704890 9025 external_mini_cluster.cc:1428] Setting key c3b55299e8340e606a9dd570965c9e86
I20260929 02:59:26.718343 19387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2: Bootstrap starting.
Sep 29 02:59:26 dist-test-slave-m1rv krb5kdc[19307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:59:26.721944 19387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2: Neither blocks nor log segments found. Creating new log.
I20260929 02:59:26.723055 19387 log.cc:824] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2: Log is configured to *not* fsync() on all Append() calls
I20260929 02:59:26.726239 19387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2: No bootstrap required, opened a new log
I20260929 02:59:26.727276 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:26.707296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.8.208.126:37039)
0929 02:59:26.708108 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:26.708119 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:26.708158 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:59:26.709839 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:26.709859 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:26.710226 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:26.710649 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:26.712126 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:26.713493 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:26.714756 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:26.715467 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:26.715682 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:26.719938 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:26.719945 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:26.719965 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:26.720012 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:26.723092 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:26.724020 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:26.724030 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:26.724038 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:26.724256 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:26.724741 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:26.724744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:26.724746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:26.725293 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:26.725508 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:26.725961 (+ 453us) server_negotiation.cc:299] Negotiation successful
0929 02:59:26.726299 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":372,"threads_started":1}
I20260929 02:59:26.733407 19387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e7892cd5ad442aa10d9b2d5de36ee2" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37039 } }
I20260929 02:59:26.733758 19387 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:59:26.733861 19387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e7892cd5ad442aa10d9b2d5de36ee2, State: Initialized, Role: FOLLOWER
I20260929 02:59:26.734388 19387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e7892cd5ad442aa10d9b2d5de36ee2" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37039 } }
I20260929 02:59:26.734622 19387 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:59:26.734761 19387 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:59:26.734925 19387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:59:26.737089 19387 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e7892cd5ad442aa10d9b2d5de36ee2" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37039 } }
I20260929 02:59:26.737591 19387 leader_election.cc:302] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68e7892cd5ad442aa10d9b2d5de36ee2; no voters:
I20260929 02:59:26.738142 19387 leader_election.cc:288] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:59:26.738274 19392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:59:26.738972 19392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [term 1 LEADER]: Becoming Leader. State: Replica: 68e7892cd5ad442aa10d9b2d5de36ee2, State: Running, Role: LEADER
I20260929 02:59:26.739514 19392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e7892cd5ad442aa10d9b2d5de36ee2" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37039 } }
I20260929 02:59:26.740096 19387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:59:26.742920 19393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68e7892cd5ad442aa10d9b2d5de36ee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e7892cd5ad442aa10d9b2d5de36ee2" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37039 } } }
I20260929 02:59:26.742895 19394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68e7892cd5ad442aa10d9b2d5de36ee2. Latest consensus state: current_term: 1 leader_uuid: "68e7892cd5ad442aa10d9b2d5de36ee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68e7892cd5ad442aa10d9b2d5de36ee2" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37039 } } }
I20260929 02:59:26.743284 19393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [sys.catalog]: This master's current role is: LEADER
I20260929 02:59:26.743284 19394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2 [sys.catalog]: This master's current role is: LEADER
I20260929 02:59:26.746776 19398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:59:26.751825 19398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:59:26.760579 19398 catalog_manager.cc:1347] Generated new cluster ID: 83e9ac9415b140619e6eeca35ac1be9a
I20260929 02:59:26.760787 19398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:59:26.776880 19398 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:59:26.778194 19398 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:59:26.791177 19398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2: Generated new TSK 0
I20260929 02:59:26.792383 19398 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 02:59:26.850569 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37039
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:59:27.195520 19413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:27.195824 19413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:27.195933 19413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:27.203438 19413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:27.203580 19413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:27.203703 19413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 02:59:27.213584 19413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37039
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:27.214857 19413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:27.216216 19413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:27.229565 19419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:27.229601 19421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:27.229647 19418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:27.234755 19413 server_base.cc:1034] running on GCE node
I20260929 02:59:27.235368 19413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:27.236296 19413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:27.237516 19413 hybrid_clock.cc:648] HybridClock initialized: now 1790650767237466 us; error 47 us; skew 500 ppm
I20260929 02:59:27.237830 19413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:59:27 dist-test-slave-m1rv krb5kdc[19307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650767, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:59:27.240911 19413 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 02:59:27.242478 19413 webserver.cc:460] Webserver started at http://127.8.208.65:36179/ using document root <none> and password file <none>
I20260929 02:59:27.243119 19413 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:27.243213 19413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:27.243455 19413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:27.246291 19413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "f094404243074a3c936fbcdfd6a7f7b5"
format_stamp: "Formatted at 2026-09-29 02:59:27 on dist-test-slave-m1rv"
server_key: "518ffdf2d28024522b26ebedfb6dd9b5"
server_key_iv: "a64d6d3766f41fb74384c79bcaddaf3d"
server_key_version: "encryptionkey@0"
I20260929 02:59:27.247047 19413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "f094404243074a3c936fbcdfd6a7f7b5"
format_stamp: "Formatted at 2026-09-29 02:59:27 on dist-test-slave-m1rv"
server_key: "518ffdf2d28024522b26ebedfb6dd9b5"
server_key_iv: "a64d6d3766f41fb74384c79bcaddaf3d"
server_key_version: "encryptionkey@0"
I20260929 02:59:27.251995 19413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:59:27.255602 19428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:27.256860 19413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 02:59:27.257015 19413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "f094404243074a3c936fbcdfd6a7f7b5"
format_stamp: "Formatted at 2026-09-29 02:59:27 on dist-test-slave-m1rv"
server_key: "518ffdf2d28024522b26ebedfb6dd9b5"
server_key_iv: "a64d6d3766f41fb74384c79bcaddaf3d"
server_key_version: "encryptionkey@0"
I20260929 02:59:27.257257 19413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:27.272701 19413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:27.276547 19413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:27.276831 19413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:27.277827 19413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:27.279150 19413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:27.279230 19413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:27.279397 19413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:27.279462 19413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:27.312556 19413 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:36349
I20260929 02:59:27.312603 19540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:36349 every 8 connection(s)
I20260929 02:59:27.313923 19413 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 02:59:27.315284 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 19413
I20260929 02:59:27.315536 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 02:59:27.316283 9025 external_mini_cluster.cc:1428] Setting key 7ba5d7d8f8aa0e78010cc1c7d147f39f
Sep 29 02:59:27 dist-test-slave-m1rv krb5kdc[19307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650767, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:59:27.335510 19543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:27.316675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:46385 (local address 127.8.208.126:37039)
0929 02:59:27.317138 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:27.317145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:27.318212 (+ 1067us) server_negotiation.cc:407] Connection header received
0929 02:59:27.320040 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:27.320045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:27.320134 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:27.320264 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:27.321825 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:27.322530 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:27.324106 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:27.324821 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:27.324964 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:27.328376 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:27.328385 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:27.328393 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:27.328438 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:27.332088 (+ 3650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:27.332837 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:27.332846 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:27.332852 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:27.333006 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:27.333485 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:27.333493 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:27.333500 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:27.333906 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:27.334066 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:27.334758 (+ 692us) server_negotiation.cc:299] Negotiation successful
0929 02:59:27.335017 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":246,"threads_started":1}
I20260929 02:59:27.336186 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:27.317256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37039 (local address 127.8.208.65:46385)
0929 02:59:27.318045 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:27.318095 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:27.319762 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:27.320502 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:27.320511 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:27.321002 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:27.321644 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:27.321667 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:27.322708 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:27.322712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:27.323937 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:27.323959 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:27.325043 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:27.325048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:27.325327 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:27.326043 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:27.326077 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:27.328149 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:59:27.332241 (+ 4092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:59:27.332248 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:59:27.332261 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:59:27.332697 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:59:27.333112 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:59:27.333115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:59:27.333118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:59:27.333350 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:59:27.334031 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:59:27.334037 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:59:27.334350 (+ 313us) client_negotiation.cc:769] Sending connection context
0929 02:59:27.334581 (+ 231us) client_negotiation.cc:240] Negotiation successful
0929 02:59:27.335004 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":296,"threads_started":1}
I20260929 02:59:27.337853 19541 heartbeater.cc:347] Connected to a master server at 127.8.208.126:37039
I20260929 02:59:27.338204 19541 heartbeater.cc:464] Registering TS with master...
I20260929 02:59:27.339252 19541 heartbeater.cc:511] Master 127.8.208.126:37039 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
I20260929 02:59:27.341864 19352 ts_manager.cc:194] Registered new tserver with Master: f094404243074a3c936fbcdfd6a7f7b5 (127.8.208.65:36349)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
I20260929 02:59:27.344041 19352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:46385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 02:59:27.370217 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37039
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:59:27.696849 19547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:27.697196 19547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:27.697378 19547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:27.705114 19547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:27.705330 19547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:27.705622 19547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 02:59:27.716706 19547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37039
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:27.718181 19547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:27.719609 19547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:27.733053 19553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:27.732996 19552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:27.733078 19555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:27.734129 19547 server_base.cc:1034] running on GCE node
I20260929 02:59:27.734727 19547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:27.735700 19547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:27.736901 19547 hybrid_clock.cc:648] HybridClock initialized: now 1790650767736853 us; error 43 us; skew 500 ppm
I20260929 02:59:27.737274 19547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:59:27 dist-test-slave-m1rv krb5kdc[19307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650767, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:59:27.740324 19547 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 02:59:27.741976 19547 webserver.cc:460] Webserver started at http://127.8.208.66:37599/ using document root <none> and password file <none>
I20260929 02:59:27.742666 19547 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:27.742815 19547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:27.743124 19547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:27.746124 19547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "2e0d6cecd6924a579bfaa23a70fab275"
format_stamp: "Formatted at 2026-09-29 02:59:27 on dist-test-slave-m1rv"
server_key: "b9e331703e2dc6e47b4f32873ddabbc7"
server_key_iv: "6a7171989456aebcbd47da6a41e0cf8c"
server_key_version: "encryptionkey@0"
I20260929 02:59:27.747077 19547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "2e0d6cecd6924a579bfaa23a70fab275"
format_stamp: "Formatted at 2026-09-29 02:59:27 on dist-test-slave-m1rv"
server_key: "b9e331703e2dc6e47b4f32873ddabbc7"
server_key_iv: "6a7171989456aebcbd47da6a41e0cf8c"
server_key_version: "encryptionkey@0"
I20260929 02:59:27.752348 19547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 02:59:27.755882 19562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:27.757257 19547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:59:27.757432 19547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "2e0d6cecd6924a579bfaa23a70fab275"
format_stamp: "Formatted at 2026-09-29 02:59:27 on dist-test-slave-m1rv"
server_key: "b9e331703e2dc6e47b4f32873ddabbc7"
server_key_iv: "6a7171989456aebcbd47da6a41e0cf8c"
server_key_version: "encryptionkey@0"
I20260929 02:59:27.757637 19547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:27.798265 19547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:27.802232 19547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:27.802548 19547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:27.803498 19547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:27.804793 19547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:27.804898 19547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:27.805058 19547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:27.805140 19547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:27.838035 19547 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:46761
I20260929 02:59:27.838124 19674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:46761 every 8 connection(s)
I20260929 02:59:27.839352 19547 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 02:59:27.841871 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 19547
I20260929 02:59:27.842079 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 02:59:27.842538 9025 external_mini_cluster.cc:1428] Setting key 93c91b5a1407ecce516518ad17f091ed
Sep 29 02:59:27 dist-test-slave-m1rv krb5kdc[19307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650767, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:59:27.863078 19678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:27.843484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:60979 (local address 127.8.208.126:37039)
0929 02:59:27.845561 (+ 2077us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:27.845568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:27.845591 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:59:27.846315 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:27.846322 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:27.846414 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:27.846541 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:27.848138 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:27.848922 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:27.850621 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:27.851352 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:27.851512 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:27.855314 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:27.855324 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:27.855332 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:27.855380 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:27.859264 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:27.860022 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:27.860030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:27.860036 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:27.860209 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:27.860621 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:27.860629 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:27.860636 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:27.861193 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:27.861366 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:27.862167 (+ 801us) server_negotiation.cc:299] Negotiation successful
0929 02:59:27.862436 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1955,"thread_start_us":1831,"threads_started":1}
I20260929 02:59:27.863910 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:27.842742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37039 (local address 127.8.208.66:60979)
0929 02:59:27.843583 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:27.843633 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:27.846038 (+ 2405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:27.846834 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:27.846844 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:27.847374 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:27.847955 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:27.847975 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:27.849091 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:27.849099 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:27.850439 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:27.850453 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:27.851584 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:27.851589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:27.851795 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:27.852472 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:27.852494 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:27.855052 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:59:27.859433 (+ 4381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:59:27.859440 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:59:27.859453 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:59:27.859880 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:59:27.860330 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:59:27.860333 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:59:27.860335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:59:27.860519 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:59:27.861322 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:59:27.861328 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:59:27.861743 (+ 415us) client_negotiation.cc:769] Sending connection context
0929 02:59:27.861988 (+ 245us) client_negotiation.cc:240] Negotiation successful
0929 02:59:27.862633 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":302,"threads_started":1}
I20260929 02:59:27.865487 19675 heartbeater.cc:347] Connected to a master server at 127.8.208.126:37039
I20260929 02:59:27.865886 19675 heartbeater.cc:464] Registering TS with master...
I20260929 02:59:27.866910 19675 heartbeater.cc:511] Master 127.8.208.126:37039 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
I20260929 02:59:27.868713 19352 ts_manager.cc:194] Registered new tserver with Master: 2e0d6cecd6924a579bfaa23a70fab275 (127.8.208.66:46761)
I20260929 02:59:27.869585 19352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:60979
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 02:59:27.900657 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37039
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:59:28.217748 19681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:59:28.218075 19681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:59:28.218228 19681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:59:28.225637 19681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:59:28.225775 19681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:59:28.225955 19681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 02:59:28.235550 19681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37039
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 02:59:28.236828 19681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:59:28.238204 19681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:59:28.251406 19686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:28.251492 19689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:59:28.251487 19687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:59:28.252070 19681 server_base.cc:1034] running on GCE node
I20260929 02:59:28.252683 19681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:59:28.253659 19681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:59:28.254882 19681 hybrid_clock.cc:648] HybridClock initialized: now 1790650768254849 us; error 46 us; skew 500 ppm
I20260929 02:59:28.255190 19681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:59:28 dist-test-slave-m1rv krb5kdc[19307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650768, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:59:28.258320 19681 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 02:59:28.259685 19681 webserver.cc:460] Webserver started at http://127.8.208.67:42415/ using document root <none> and password file <none>
I20260929 02:59:28.260406 19681 fs_manager.cc:359] Metadata directory not provided
I20260929 02:59:28.260550 19681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:59:28.260871 19681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:59:28.264158 19681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "b97cf3eda2994d21b41f9298f95965fa"
format_stamp: "Formatted at 2026-09-29 02:59:28 on dist-test-slave-m1rv"
server_key: "a61dc96cf7972227034a67192712ecd2"
server_key_iv: "a6e439c97426f2b8b4cba60cad5ac1f4"
server_key_version: "encryptionkey@0"
I20260929 02:59:28.264912 19681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "b97cf3eda2994d21b41f9298f95965fa"
format_stamp: "Formatted at 2026-09-29 02:59:28 on dist-test-slave-m1rv"
server_key: "a61dc96cf7972227034a67192712ecd2"
server_key_iv: "a6e439c97426f2b8b4cba60cad5ac1f4"
server_key_version: "encryptionkey@0"
I20260929 02:59:28.270406 19681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:59:28.273996 19696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:28.275308 19681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:59:28.275483 19681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "b97cf3eda2994d21b41f9298f95965fa"
format_stamp: "Formatted at 2026-09-29 02:59:28 on dist-test-slave-m1rv"
server_key: "a61dc96cf7972227034a67192712ecd2"
server_key_iv: "a6e439c97426f2b8b4cba60cad5ac1f4"
server_key_version: "encryptionkey@0"
I20260929 02:59:28.275686 19681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:59:28.290648 19681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:59:28.294428 19681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:59:28.294700 19681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:59:28.295579 19681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:59:28.296851 19681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:59:28.296952 19681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:28.297101 19681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:59:28.297173 19681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:59:28.331988 19681 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:39843
I20260929 02:59:28.332022 19808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:39843 every 8 connection(s)
I20260929 02:59:28.333362 19681 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 02:59:28.341142 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 19681
I20260929 02:59:28.341404 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 02:59:28.341817 9025 external_mini_cluster.cc:1428] Setting key 8c37e346ddbd080d29604d330d38c6f8
Sep 29 02:59:28 dist-test-slave-m1rv krb5kdc[19307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650768, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 02:59:28.347476 19541 heartbeater.cc:503] Master 127.8.208.126:37039 was elected leader, sending a full tablet report...
I20260929 02:59:28.352932 19678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.335872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:59433 (local address 127.8.208.126:37039)
0929 02:59:28.336097 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:28.336104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:28.337374 (+ 1270us) server_negotiation.cc:407] Connection header received
0929 02:59:28.339077 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:28.339081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:28.339163 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:28.339256 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:28.340585 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.341646 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.343430 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.343977 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.344105 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.347478 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:28.347485 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:28.347489 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:28.347529 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:28.350388 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:28.350991 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:28.350995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:28.350997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:28.351103 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:28.351553 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:28.351556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:28.351558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:28.351808 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:28.351931 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:28.352554 (+ 623us) server_negotiation.cc:299] Negotiation successful
0929 02:59:28.352709 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 02:59:28.354066 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.336452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37039 (local address 127.8.208.67:59433)
0929 02:59:28.337153 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:28.337268 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:28.338797 (+ 1529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:28.339447 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:28.339454 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:28.339934 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:59:28.340447 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:28.340463 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.341806 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.341810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:28.343290 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:28.343301 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.344167 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.344172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:28.344366 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.345117 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:59:28.345137 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:59:28.347274 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:59:28.350502 (+ 3228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:59:28.350507 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:59:28.350518 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:59:28.350882 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:59:28.351236 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:59:28.351240 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:59:28.351242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:59:28.351451 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:59:28.351931 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:59:28.351936 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:59:28.352234 (+ 298us) client_negotiation.cc:769] Sending connection context
0929 02:59:28.352480 (+ 246us) client_negotiation.cc:240] Negotiation successful
0929 02:59:28.352898 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":275,"threads_started":1}
I20260929 02:59:28.355646 19809 heartbeater.cc:347] Connected to a master server at 127.8.208.126:37039
I20260929 02:59:28.355997 19809 heartbeater.cc:464] Registering TS with master...
I20260929 02:59:28.356877 19809 heartbeater.cc:511] Master 127.8.208.126:37039 requested a full tablet report, sending...
I20260929 02:59:28.358585 19352 ts_manager.cc:194] Registered new tserver with Master: b97cf3eda2994d21b41f9298f95965fa (127.8.208.67:39843)
I20260929 02:59:28.359481 19352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:59433
I20260929 02:59:28.370085 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:59:28.386682 19678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.374585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49802 (local address 127.8.208.126:37039)
0929 02:59:28.374749 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:28.374754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:28.375205 (+ 451us) server_negotiation.cc:407] Connection header received
0929 02:59:28.375339 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:28.375343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:28.375414 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:28.375495 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:28.376775 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.377673 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.378952 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.379428 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.379562 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.381765 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:28.381770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:28.381773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:28.381803 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:28.384328 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:28.384883 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:28.384887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:28.384890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:28.385007 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:28.385340 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:28.385344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:28.385346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:28.385666 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:28.385872 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:28.386130 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 02:59:28.386341 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:59:28.390743 19352 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:59:28.397498 19352 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:59:28.431408 19822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.409281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:46761 (local address 127.0.0.1:33574)
0929 02:59:28.410725 (+ 1444us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:28.410742 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:28.410851 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:28.424205 (+ 13354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:28.424211 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:28.424241 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:59:28.424685 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:28.424702 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.426320 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.426328 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:28.428591 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:28.428606 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.430402 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.430408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:28.430579 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.430639 (+ 60us) client_negotiation.cc:769] Sending connection context
0929 02:59:28.430775 (+ 136us) client_negotiation.cc:240] Negotiation successful
0929 02:59:28.430975 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1334,"mutex_wait_us":3,"thread_start_us":2408,"threads_started":1}
I20260929 02:59:28.431490 19825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.411000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:36349 (local address 127.0.0.1:47002)
0929 02:59:28.423303 (+ 12303us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:28.423322 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:28.423434 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:28.424630 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:28.424634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:28.424663 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:59:28.425075 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:28.425089 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.427204 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.427210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:28.429222 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:28.429238 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.430738 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.430747 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:28.430909 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.430938 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 02:59:28.431007 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 02:59:28.431121 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12171,"mutex_wait_us":1,"thread_start_us":12954,"threads_started":1}
I20260929 02:59:28.431529 19824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.409489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33574 (local address 127.8.208.66:46761)
0929 02:59:28.423028 (+ 13539us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:28.423042 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:28.423082 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:59:28.423222 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:28.423234 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:28.423567 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:28.424148 (+ 581us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:59:28.424912 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.426181 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.428744 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.430212 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.430367 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.430584 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:28.431004 (+ 420us) server_negotiation.cc:299] Negotiation successful
0929 02:59:28.431178 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13299,"thread_start_us":14265,"threads_started":1}
I20260929 02:59:28.432230 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.411459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.8.208.65:36349)
0929 02:59:28.423982 (+ 12523us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:28.423990 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:28.424018 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:59:28.424147 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:28.424155 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:28.424397 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:28.424622 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:59:28.425364 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.427056 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.429364 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.430524 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.430651 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.430870 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:28.431691 (+ 821us) server_negotiation.cc:299] Negotiation successful
0929 02:59:28.431872 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12356,"thread_start_us":12243,"threads_started":1}
I20260929 02:59:28.432931 19821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.409041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:39843 (local address 127.0.0.1:35054)
0929 02:59:28.409967 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:28.410016 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:28.410224 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:28.425107 (+ 14883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:28.425114 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:28.425145 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:59:28.425601 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:28.425616 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.427597 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.427602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:28.429861 (+ 2259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:28.429875 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.432145 (+ 2270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.432151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:28.432341 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.432377 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 02:59:28.432462 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 02:59:28.432580 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":1,"thread_start_us":603,"threads_started":1}
I20260929 02:59:28.433281 19823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.409321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35054 (local address 127.8.208.67:39843)
0929 02:59:28.424499 (+ 15178us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:28.424505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:28.424529 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:59:28.424644 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:28.424650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:28.424876 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:28.425786 (+ 910us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:59:28.426041 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.427456 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.430047 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.431393 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.432066 (+ 673us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.432491 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:28.432584 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 02:59:28.432750 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15017,"thread_start_us":15515,"threads_started":1}
I20260929 02:59:28.435561 19476 tablet_service.cc:1467] Processing CreateTablet for tablet 1c78ab27dce34498b64efab3c214df93 (DEFAULT_TABLE table=test-table [id=4f8155d092d44240a7208ff5ff0d8281]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:59:28.435606 19610 tablet_service.cc:1467] Processing CreateTablet for tablet 1c78ab27dce34498b64efab3c214df93 (DEFAULT_TABLE table=test-table [id=4f8155d092d44240a7208ff5ff0d8281]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:59:28.436971 19476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c78ab27dce34498b64efab3c214df93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:59:28.437031 19610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c78ab27dce34498b64efab3c214df93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:59:28.436625 19744 tablet_service.cc:1467] Processing CreateTablet for tablet 1c78ab27dce34498b64efab3c214df93 (DEFAULT_TABLE table=test-table [id=4f8155d092d44240a7208ff5ff0d8281]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:59:28.437877 19744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c78ab27dce34498b64efab3c214df93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:59:28.448449 19827 tablet_bootstrap.cc:492] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5: Bootstrap starting.
I20260929 02:59:28.451489 19828 tablet_bootstrap.cc:492] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275: Bootstrap starting.
I20260929 02:59:28.452579 19827 tablet_bootstrap.cc:654] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5: Neither blocks nor log segments found. Creating new log.
I20260929 02:59:28.453213 19829 tablet_bootstrap.cc:492] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa: Bootstrap starting.
I20260929 02:59:28.453904 19827 log.cc:824] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:59:28.455215 19828 tablet_bootstrap.cc:654] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275: Neither blocks nor log segments found. Creating new log.
I20260929 02:59:28.456429 19828 log.cc:824] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275: Log is configured to *not* fsync() on all Append() calls
I20260929 02:59:28.456501 19829 tablet_bootstrap.cc:654] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa: Neither blocks nor log segments found. Creating new log.
I20260929 02:59:28.457286 19827 tablet_bootstrap.cc:492] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5: No bootstrap required, opened a new log
I20260929 02:59:28.457547 19827 ts_tablet_manager.cc:1397] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260929 02:59:28.457697 19829 log.cc:824] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa: Log is configured to *not* fsync() on all Append() calls
I20260929 02:59:28.459434 19828 tablet_bootstrap.cc:492] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275: No bootstrap required, opened a new log
I20260929 02:59:28.459733 19828 ts_tablet_manager.cc:1397] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260929 02:59:28.460557 19829 tablet_bootstrap.cc:492] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa: No bootstrap required, opened a new log
I20260929 02:59:28.461128 19829 ts_tablet_manager.cc:1397] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260929 02:59:28.468158 19829 raft_consensus.cc:348] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97cf3eda2994d21b41f9298f95965fa" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39843 } } peers { permanent_uuid: "2e0d6cecd6924a579bfaa23a70fab275" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46761 } } peers { permanent_uuid: "f094404243074a3c936fbcdfd6a7f7b5" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 36349 } }
I20260929 02:59:28.468523 19829 raft_consensus.cc:374] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:59:28.468350 19827 raft_consensus.cc:348] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97cf3eda2994d21b41f9298f95965fa" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39843 } } peers { permanent_uuid: "2e0d6cecd6924a579bfaa23a70fab275" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46761 } } peers { permanent_uuid: "f094404243074a3c936fbcdfd6a7f7b5" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 36349 } }
I20260929 02:59:28.468617 19829 raft_consensus.cc:729] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b97cf3eda2994d21b41f9298f95965fa, State: Initialized, Role: FOLLOWER
I20260929 02:59:28.468757 19827 raft_consensus.cc:374] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:59:28.468850 19827 raft_consensus.cc:729] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f094404243074a3c936fbcdfd6a7f7b5, State: Initialized, Role: FOLLOWER
I20260929 02:59:28.469322 19829 consensus_queue.cc:260] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97cf3eda2994d21b41f9298f95965fa" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39843 } } peers { permanent_uuid: "2e0d6cecd6924a579bfaa23a70fab275" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46761 } } peers { permanent_uuid: "f094404243074a3c936fbcdfd6a7f7b5" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 36349 } }
I20260929 02:59:28.469442 19827 consensus_queue.cc:260] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97cf3eda2994d21b41f9298f95965fa" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39843 } } peers { permanent_uuid: "2e0d6cecd6924a579bfaa23a70fab275" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46761 } } peers { permanent_uuid: "f094404243074a3c936fbcdfd6a7f7b5" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 36349 } }
I20260929 02:59:28.470077 19828 raft_consensus.cc:348] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97cf3eda2994d21b41f9298f95965fa" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39843 } } peers { permanent_uuid: "2e0d6cecd6924a579bfaa23a70fab275" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46761 } } peers { permanent_uuid: "f094404243074a3c936fbcdfd6a7f7b5" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 36349 } }
I20260929 02:59:28.470469 19828 raft_consensus.cc:374] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:59:28.470569 19828 raft_consensus.cc:729] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e0d6cecd6924a579bfaa23a70fab275, State: Initialized, Role: FOLLOWER
I20260929 02:59:28.471287 19827 ts_tablet_manager.cc:1428] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260929 02:59:28.471102 19828 consensus_queue.cc:260] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97cf3eda2994d21b41f9298f95965fa" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39843 } } peers { permanent_uuid: "2e0d6cecd6924a579bfaa23a70fab275" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46761 } } peers { permanent_uuid: "f094404243074a3c936fbcdfd6a7f7b5" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 36349 } }
I20260929 02:59:28.472193 19675 heartbeater.cc:503] Master 127.8.208.126:37039 was elected leader, sending a full tablet report...
I20260929 02:59:28.473002 19828 ts_tablet_manager.cc:1428] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260929 02:59:28.476271 19809 heartbeater.cc:503] Master 127.8.208.126:37039 was elected leader, sending a full tablet report...
I20260929 02:59:28.476822 19829 ts_tablet_manager.cc:1428] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa: Time spent starting tablet: real 0.016s user 0.007s sys 0.007s
I20260929 02:59:28.564698 19835 raft_consensus.cc:482] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:59:28.565024 19835 raft_consensus.cc:504] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97cf3eda2994d21b41f9298f95965fa" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39843 } } peers { permanent_uuid: "2e0d6cecd6924a579bfaa23a70fab275" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46761 } } peers { permanent_uuid: "f094404243074a3c936fbcdfd6a7f7b5" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 36349 } }
W20260929 02:59:28.566232 19542 tablet.cc:2369] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:59:28.566879 19835 leader_election.cc:288] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b97cf3eda2994d21b41f9298f95965fa (127.8.208.67:39843), f094404243074a3c936fbcdfd6a7f7b5 (127.8.208.65:36349)
I20260929 02:59:28.573379 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.567349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:33511 (local address 127.8.208.65:36349)
0929 02:59:28.567556 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:28.567566 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:28.568156 (+ 590us) server_negotiation.cc:407] Connection header received
0929 02:59:28.568397 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:28.568407 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:28.568465 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:28.568561 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:59:28.569037 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.569793 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.571576 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.572416 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.572634 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.572712 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:28.572984 (+ 272us) server_negotiation.cc:299] Negotiation successful
0929 02:59:28.573109 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 02:59:28.573549 19836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.567348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:36349 (local address 127.8.208.66:33511)
0929 02:59:28.568045 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:28.568060 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:28.568246 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:28.568577 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:28.568581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:28.568605 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:59:28.568897 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:28.568905 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.569916 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.569921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:28.571421 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:28.571438 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.572653 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.572659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:28.572820 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.572844 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:59:28.572955 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 02:59:28.573081 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":2,"thread_start_us":361,"threads_started":1}
I20260929 02:59:28.574038 19823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.567347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:60109 (local address 127.8.208.67:39843)
0929 02:59:28.567573 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:28.567581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:28.568610 (+ 1029us) server_negotiation.cc:407] Connection header received
0929 02:59:28.568820 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:28.568825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:28.568894 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:28.568997 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:59:28.569614 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.570346 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.572149 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.573022 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.573221 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.573301 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:28.573695 (+ 394us) server_negotiation.cc:299] Negotiation successful
0929 02:59:28.573841 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 02:59:28.574059 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.567348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:39843 (local address 127.8.208.66:60109)
0929 02:59:28.568487 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0929 02:59:28.568502 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:59:28.568670 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:59:28.569021 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:59:28.569024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:59:28.569048 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:59:28.569458 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:28.569467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.570500 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.570505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:28.571997 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:59:28.572009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.573275 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.573281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:59:28.573441 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.573519 (+ 78us) client_negotiation.cc:769] Sending connection context
0929 02:59:28.573649 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 02:59:28.573789 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1040,"mutex_wait_us":237,"thread_start_us":550,"threads_started":1}
I20260929 02:59:28.575047 19496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c78ab27dce34498b64efab3c214df93" candidate_uuid: "2e0d6cecd6924a579bfaa23a70fab275" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f094404243074a3c936fbcdfd6a7f7b5" is_pre_election: true
I20260929 02:59:28.575150 19764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c78ab27dce34498b64efab3c214df93" candidate_uuid: "2e0d6cecd6924a579bfaa23a70fab275" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b97cf3eda2994d21b41f9298f95965fa" is_pre_election: true
I20260929 02:59:28.575688 19764 raft_consensus.cc:2454] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e0d6cecd6924a579bfaa23a70fab275 in term 0.
I20260929 02:59:28.575690 19496 raft_consensus.cc:2454] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e0d6cecd6924a579bfaa23a70fab275 in term 0.
I20260929 02:59:28.576385 19564 leader_election.cc:302] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e0d6cecd6924a579bfaa23a70fab275, f094404243074a3c936fbcdfd6a7f7b5; no voters:
I20260929 02:59:28.576771 19835 raft_consensus.cc:2781] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:59:28.576936 19835 raft_consensus.cc:482] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:59:28.577066 19835 raft_consensus.cc:3037] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:59:28.579211 19835 raft_consensus.cc:504] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97cf3eda2994d21b41f9298f95965fa" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39843 } } peers { permanent_uuid: "2e0d6cecd6924a579bfaa23a70fab275" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46761 } } peers { permanent_uuid: "f094404243074a3c936fbcdfd6a7f7b5" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 36349 } }
I20260929 02:59:28.579802 19835 leader_election.cc:288] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [CANDIDATE]: Term 1 election: Requested vote from peers b97cf3eda2994d21b41f9298f95965fa (127.8.208.67:39843), f094404243074a3c936fbcdfd6a7f7b5 (127.8.208.65:36349)
I20260929 02:59:28.580284 19764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c78ab27dce34498b64efab3c214df93" candidate_uuid: "2e0d6cecd6924a579bfaa23a70fab275" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b97cf3eda2994d21b41f9298f95965fa"
I20260929 02:59:28.580483 19496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c78ab27dce34498b64efab3c214df93" candidate_uuid: "2e0d6cecd6924a579bfaa23a70fab275" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f094404243074a3c936fbcdfd6a7f7b5"
I20260929 02:59:28.580538 19764 raft_consensus.cc:3037] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:59:28.580691 19496 raft_consensus.cc:3037] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:59:28.582824 19496 raft_consensus.cc:2454] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e0d6cecd6924a579bfaa23a70fab275 in term 1.
I20260929 02:59:28.582828 19764 raft_consensus.cc:2454] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e0d6cecd6924a579bfaa23a70fab275 in term 1.
I20260929 02:59:28.583253 19566 leader_election.cc:302] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e0d6cecd6924a579bfaa23a70fab275, b97cf3eda2994d21b41f9298f95965fa; no voters:
I20260929 02:59:28.583560 19835 raft_consensus.cc:2781] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:59:28.583997 19835 raft_consensus.cc:686] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 1 LEADER]: Becoming Leader. State: Replica: 2e0d6cecd6924a579bfaa23a70fab275, State: Running, Role: LEADER
I20260929 02:59:28.584442 19835 consensus_queue.cc:237] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97cf3eda2994d21b41f9298f95965fa" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39843 } } peers { permanent_uuid: "2e0d6cecd6924a579bfaa23a70fab275" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46761 } } peers { permanent_uuid: "f094404243074a3c936fbcdfd6a7f7b5" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 36349 } }
W20260929 02:59:28.584925 19810 tablet.cc:2369] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:59:28.588857 19352 catalog_manager.cc:5495] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e0d6cecd6924a579bfaa23a70fab275 (127.8.208.66). New cstate: current_term: 1 leader_uuid: "2e0d6cecd6924a579bfaa23a70fab275" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97cf3eda2994d21b41f9298f95965fa" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e0d6cecd6924a579bfaa23a70fab275" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f094404243074a3c936fbcdfd6a7f7b5" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 36349 } health_report { overall_health: UNKNOWN } } }
W20260929 02:59:28.591444 19676 tablet.cc:2369] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:59:28.612895 19824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.607691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33580 (local address 127.8.208.66:46761)
0929 02:59:28.607929 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:28.607935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:28.607965 (+ 30us) server_negotiation.cc:407] Connection header received
0929 02:59:28.608039 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:28.608043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:28.608113 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:28.608268 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:59:28.608662 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.609434 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.610758 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.611175 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.611315 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.611792 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:59:28.612290 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:59:28.612452 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:28.612550 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 02:59:28.612630 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:59:28.620443 19496 raft_consensus.cc:1261] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5 [term 1 FOLLOWER]: Refusing update from remote peer 2e0d6cecd6924a579bfaa23a70fab275: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:59:28.620486 19764 raft_consensus.cc:1261] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa [term 1 FOLLOWER]: Refusing update from remote peer 2e0d6cecd6924a579bfaa23a70fab275: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:59:28.621418 19838 consensus_queue.cc:1035] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f094404243074a3c936fbcdfd6a7f7b5" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 36349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:59:28.621811 19835 consensus_queue.cc:1035] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b97cf3eda2994d21b41f9298f95965fa" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:59:28.633595 19843 mvcc.cc:205] Tried to move back new op lower bound from 7334505548260364288 to 7334505548126261248. Current Snapshot: MvccSnapshot[applied={T|T < 7334505548260364288}]
I20260929 02:59:28.655697 19350 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:59:28.655942 19350 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:59:28.659660 19350 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68e7892cd5ad442aa10d9b2d5de36ee2: Sending DeleteTablet for 3 replicas of tablet 1c78ab27dce34498b64efab3c214df93
I20260929 02:59:28.660740 19610 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c78ab27dce34498b64efab3c214df93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:59:28 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:33574
I20260929 02:59:28.660763 19744 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c78ab27dce34498b64efab3c214df93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:59:28 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:35054
I20260929 02:59:28.661769 19852 tablet_replica.cc:330] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa: stopping tablet replica
I20260929 02:59:28.661772 19851 tablet_replica.cc:330] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275: stopping tablet replica
I20260929 02:59:28.662231 19852 raft_consensus.cc:2229] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:59:28.662220 19851 raft_consensus.cc:2229] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:59:28.662298 19476 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c78ab27dce34498b64efab3c214df93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:59:28 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:47002
I20260929 02:59:28.662927 19852 raft_consensus.cc:2258] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:59:28.662992 19851 raft_consensus.cc:2258] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:59:28.665299 19852 ts_tablet_manager.cc:1905] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:59:28.665112 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 19413
I20260929 02:59:28.672345 19852 ts_tablet_manager.cc:1918] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:59:28.672662 19852 log.cc:1196] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/1c78ab27dce34498b64efab3c214df93
I20260929 02:59:28.673300 19852 ts_tablet_manager.cc:1939] T 1c78ab27dce34498b64efab3c214df93 P b97cf3eda2994d21b41f9298f95965fa: Deleting consensus metadata
I20260929 02:59:28.674959 19853 tablet_replica.cc:330] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5: stopping tablet replica
I20260929 02:59:28.675123 19339 catalog_manager.cc:4841] TS b97cf3eda2994d21b41f9298f95965fa (127.8.208.67:39843): tablet 1c78ab27dce34498b64efab3c214df93 (table test-table [id=4f8155d092d44240a7208ff5ff0d8281]) successfully deleted
I20260929 02:59:28.675379 19853 raft_consensus.cc:2229] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:59:28.675907 19853 raft_consensus.cc:2258] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:59:28.676684 19851 ts_tablet_manager.cc:1905] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:59:28.677819 19853 ts_tablet_manager.cc:1905] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:59:28.684966 19851 ts_tablet_manager.cc:1918] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:59:28.684966 19853 ts_tablet_manager.cc:1918] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:59:28.685155 19853 log.cc:1196] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/1c78ab27dce34498b64efab3c214df93
I20260929 02:59:28.685199 19851 log.cc:1196] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/1c78ab27dce34498b64efab3c214df93
I20260929 02:59:28.685783 19853 ts_tablet_manager.cc:1939] T 1c78ab27dce34498b64efab3c214df93 P f094404243074a3c936fbcdfd6a7f7b5: Deleting consensus metadata
I20260929 02:59:28.685797 19851 ts_tablet_manager.cc:1939] T 1c78ab27dce34498b64efab3c214df93 P 2e0d6cecd6924a579bfaa23a70fab275: Deleting consensus metadata
I20260929 02:59:28.687741 19336 catalog_manager.cc:4841] TS 2e0d6cecd6924a579bfaa23a70fab275 (127.8.208.66:46761): tablet 1c78ab27dce34498b64efab3c214df93 (table test-table [id=4f8155d092d44240a7208ff5ff0d8281]) successfully deleted
I20260929 02:59:28.687848 19337 catalog_manager.cc:4841] TS f094404243074a3c936fbcdfd6a7f7b5 (127.8.208.65:36349): tablet 1c78ab27dce34498b64efab3c214df93 (table test-table [id=4f8155d092d44240a7208ff5ff0d8281]) successfully deleted
Sep 29 02:59:28 dist-test-slave-m1rv krb5kdc[19307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 02:59:28.700037 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.668410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47014 (local address 127.8.208.65:36349)
0929 02:59:28.668628 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:28.668638 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:28.681706 (+ 13068us) server_negotiation.cc:407] Connection header received
0929 02:59:28.682007 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:28.682013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:28.682106 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:28.682184 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:28.683971 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.685158 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.687283 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.687935 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.688044 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.693068 (+ 5024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:28.693085 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:28.693103 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:28.693142 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:28.697000 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:28.697754 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:28.697764 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:28.697768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:28.697927 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:28.698298 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:28.698304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:28.698307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:28.698749 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:28.699079 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:28.699398 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 02:59:28.699619 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:59:28.701129 19536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:28.800331 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 19413
I20260929 02:59:28.817553 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 19547
Sep 29 02:59:28 dist-test-slave-m1rv krb5kdc[19307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 02:59:28.835251 19824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.818365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.8.208.66:46761)
0929 02:59:28.818570 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:28.818577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:28.818607 (+ 30us) server_negotiation.cc:407] Connection header received
0929 02:59:28.818752 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:28.818759 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:28.818852 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:28.818983 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:28.820161 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.821482 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.822904 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.823621 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.823770 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.828028 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:28.828047 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:28.828069 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:28.828128 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:28.832299 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:28.833026 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:28.833037 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:28.833041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:28.833241 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:28.833583 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:28.833588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:28.833592 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:28.834061 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:28.834420 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:28.834590 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 02:59:28.834820 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 02:59:28.836069 19670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:28.956489 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 19547
I20260929 02:59:28.975809 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 19681
Sep 29 02:59:28 dist-test-slave-m1rv krb5kdc[19307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 02:59:28.990175 19823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:59:28.976568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.8.208.67:39843)
0929 02:59:28.976754 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:59:28.976759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:59:28.976777 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:59:28.976906 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:59:28.976910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:59:28.976973 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:59:28.977066 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:59:28.978353 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.979108 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.980382 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:59:28.980760 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:59:28.980866 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:59:28.984947 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:59:28.984961 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:59:28.984977 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:59:28.985006 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:59:28.987573 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:28.988208 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:28.988214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:28.988216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:28.988349 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:59:28.988662 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:59:28.988665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:59:28.988667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:59:28.989032 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:59:28.989558 (+ 526us) server_negotiation.cc:1035] Waiting for connection context
0929 02:59:28.989684 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 02:59:28.989899 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 02:59:28.990810 19804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:29.080032 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 19681
I20260929 02:59:29.096195 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 19320
I20260929 02:59:29.097440 19382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:59:29.206132 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 19320
2026-09-29T02:59:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4275 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:59:29 dist-test-slave-m1rv krb5kdc[19861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:59:29 dist-test-slave-m1rv krb5kdc[19861](info): set up 2 sockets
Sep 29 02:59:29 dist-test-slave-m1rv krb5kdc[19861](info): commencing operation
krb5kdc: starting...
W20260929 02:59:30.310114 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 02:59:30 dist-test-slave-m1rv krb5kdc[19861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:59:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:59:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.208.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.208.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/rangeradmin_127.8.208.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/rangeradmin_127.8.208.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.208.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.208.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/rangerlookup_127.8.208.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/rangerlookup_127.8.208.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.208.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.208.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/HTTP_127.8.208.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/HTTP_127.8.208.84@KRBTEST.COM.keytab.
I20260929 02:59:30.616478 9025 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1CUkeB/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/postgres -l logfile start
2026-09-29 02:59:33.191 UTC [19891] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 02:59:33.191 UTC [19891] LOG: listening on IPv4 address "127.8.208.84", port 54851
2026-09-29 02:59:33.195 UTC [19891] LOG: listening on Unix socket "/tmp/.s.PGSQL.54851"
2026-09-29 02:59:33.216 UTC [19894] LOG: database system was shut down at 2026-09-29 02:59:31 UTC
2026-09-29 02:59:33.219 UTC [19891] LOG: database system is ready to accept connections
I20260929 02:59:34.197835 9025 mini_postgres.cc:96] Postgres bound to 54851
2026-09-29 02:59:34.214 UTC [19902] FATAL: database "slave" does not exist
127.8.208.84:54851 - accepting connections
I20260929 02:59:34.215718 9025 mini_ranger.cc:161] Starting Ranger...
I20260929 02:59:34.241394 9025 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 02:59:34.749163 9025 mini_ranger.cc:87] Created ranger Postgres database
I20260929 02:59:34.749327 9025 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-admin
2026-09-29 02:59:35,048 [I] DB FLAVOR :POSTGRES
2026-09-29 02:59:35,049 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 02:59:35,049 [I] Checking connection..
2026-09-29 02:59:35,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 02:59:35,305 [I] Checking connection passed.
2026-09-29 02:59:35,305 [I] --------- Verifying version history table ---------
2026-09-29 02:59:35,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:59:35,562 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 02:59:35,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:59:35,789 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 02:59:35,789 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 02:59:35,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 02:59:36.017 UTC [19966] WARNING: there is no transaction in progress
2026-09-29 02:59:36,023 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 02:59:36,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:59:36,281 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 02:59:36,282 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 02:59:36,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 02:59:36,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 02:59:36,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-m1rv.c.gcp-upstream.internal','N') ;"
2026-09-29 02:59:36,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 02:59:37,227 [I] Table x_portal_user does not exist in database ranger
2026-09-29 02:59:37,228 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 02:59:37,487 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 02:59:37,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 02:59:37,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 02:59:37,981 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 02:59:37,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 02:59:38.943 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:38.959 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:38.976 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:38.992 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39.008 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39.025 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39.079 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39.086 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39.100 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39.110 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39.121 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39.135 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39.145 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39.151 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39.158 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39.166 UTC [20074] WARNING: there is no transaction in progress
2026-09-29 02:59:39,801 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 02:59:39,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 02:59:40,061 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 02:59:40,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 02:59:40,301 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 02:59:40,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 02:59:40,561 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 02:59:40,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 02:59:40,818 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 02:59:41,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-m1rv.c.gcp-upstream.internal';"
2026-09-29 02:59:41,320 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 02:59:41,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 02:59:41,543 [I] DB_PATCHES have already been applied
I20260929 02:59:41.549295 9025 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-admin:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1CUkeB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 02:59:41.549414 9025 mini_ranger.cc:193] Using host: 127.8.208.84
I20260929 02:59:41.563661 9025 mini_ranger.cc:227] Ranger admin URL: http://127.8.208.84:38013
Sep 29, 2026 2:59:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 2:59:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 2:59:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 2:59:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 2:59:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 2:59:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 2:59:42 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 02:59:42.692562 9025 mini_ranger.cc:160] Time spent starting Ranger: real 8.477s user 0.012s sys 0.025s
Sep 29, 2026 3:00:02 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 3:00:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 3:00:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 3:00:03 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 3:00:03 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 03:00:04.558151 9025 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.208.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.208.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/rangerkms_127.8.208.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/rangerkms_127.8.208.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.208.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.208.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.208.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/HTTP_127.8.208.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.208.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/HTTP_127.8.208.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 03:00:04 dist-test-slave-m1rv krb5kdc[19861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650804, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 03:00:04.745087 9025 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 03:00:04.770964 9025 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 03:00:05.641638 9025 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 03:00:05.641786 9025 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-kms
I20260929 03:00:05.641953 9025 mini_ranger_kms.cc:227] Using postgres at 127.8.208.84:54851
2026-09-29 03:00:06,065 [I] DB FLAVOR :POSTGRES
2026-09-29 03:00:06,065 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 03:00:06,066 [I] Checking connection
2026-09-29 03:00:06,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 03:00:06,291 [I] connection success
2026-09-29 03:00:06,291 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 03:00:06,291 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 03:00:06,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 03:00:06,525 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 03:00:06,526 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 03:00:06,526 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 03:00:06,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:54851/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 03:00:06,791 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 03:00:07.246305 9025 mini_ranger_kms.cc:310] Created kms service
I20260929 03:00:07.448272 9025 mini_ranger_kms.cc:326] Created kudu user
I20260929 03:00:07.495932 9025 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 03:00:07 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650804, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:00:07.915400 9025 mini_ranger_kms.cc:384] Added ranger policy
I20260929 03:00:07.915817 9025 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-kms:/tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 03:00:07.915885 9025 mini_ranger_kms.cc:240] Using host: 127.8.208.84
I20260929 03:00:07.937817 9025 mini_ranger_kms.cc:276] Ranger KMS PID: 20254
I20260929 03:00:07.938035 9025 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.208.84:46189
Sep 29, 2026 3:00:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 3:00:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 3:00:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 3:00:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 3:00:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 3:00:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 29 03:00:10 dist-test-slave-m1rv krb5kdc[19861](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790650810, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.208.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:00:10.331972 9025 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.587s user 0.145s sys 0.300s
I20260929 03:00:10.332216 9025 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 03:00:10.332438 9025 mini_ranger_kms.cc:401] 127.8.208.84:46189/kms/v1/keys
Sep 29 03:00:10 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650810, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.208.84@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
Sep 29 03:00:11 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650810, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.208.84@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
Sep 29 03:00:13 dist-test-slave-m1rv krb5kdc[19861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:00:13.321631 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:40907
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:40907
--ranger_config_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.208.84:46189/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:00:13.650197 20305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:00:13.650516 20305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:00:13.650698 20305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:00:13.658006 20305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:00:13.658154 20305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:00:13.658275 20305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:00:13.658382 20305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:00:13.658479 20305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:00:13.668864 20305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.208.84:46189/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:40907
--ranger_config_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:40907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:00:13.670182 20305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:00:13.671483 20305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:00:13.683892 20313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:00:13.683946 20310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:00:13.683969 20311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:00:13.684607 20305 server_base.cc:1034] running on GCE node
I20260929 03:00:13.685334 20305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:00:13.686632 20305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:00:13.687821 20305 hybrid_clock.cc:648] HybridClock initialized: now 1790650813687767 us; error 50 us; skew 500 ppm
I20260929 03:00:13.688107 20305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:00:13 dist-test-slave-m1rv krb5kdc[19861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650813, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:00:13.691068 20305 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:00:13.692312 20305 webserver.cc:460] Webserver started at http://127.8.208.126:34473/ using document root <none> and password file <none>
I20260929 03:00:13.692893 20305 fs_manager.cc:359] Metadata directory not provided
I20260929 03:00:13.692978 20305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:00:13.693265 20305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:00:13 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650813, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:00:13.740492 20305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "2fa8d3fb57954c0c88d8d0d5b8ba2583"
format_stamp: "Formatted at 2026-09-29 03:00:13 on dist-test-slave-m1rv"
server_key: "f1bfb6cf5d2ec2c776647cbb98afd97e"
server_key_iv: "ba0b335e2375049e3a5e15fec293bede"
server_key_version: "kuduclusterkey@0"
I20260929 03:00:13.741299 20305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "2fa8d3fb57954c0c88d8d0d5b8ba2583"
format_stamp: "Formatted at 2026-09-29 03:00:13 on dist-test-slave-m1rv"
server_key: "f1bfb6cf5d2ec2c776647cbb98afd97e"
server_key_iv: "ba0b335e2375049e3a5e15fec293bede"
server_key_version: "kuduclusterkey@0"
I20260929 03:00:13.746742 20305 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260929 03:00:13.764768 20320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:00:13.766899 20305 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:00:13.767102 20305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "2fa8d3fb57954c0c88d8d0d5b8ba2583"
format_stamp: "Formatted at 2026-09-29 03:00:13 on dist-test-slave-m1rv"
server_key: "f1bfb6cf5d2ec2c776647cbb98afd97e"
server_key_iv: "ba0b335e2375049e3a5e15fec293bede"
server_key_version: "kuduclusterkey@0"
I20260929 03:00:13.767311 20305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:00:13.791177 20305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:00:13.795086 20305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:00:13.795382 20305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:00:13.814688 20305 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:40907
I20260929 03:00:13.814711 20371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:40907 every 8 connection(s)
I20260929 03:00:13.815989 20305 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:00:13.817677 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 20305
I20260929 03:00:13.817890 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:00:13.818704 9025 external_mini_cluster.cc:1428] Setting key db959ce57704e8ed5c4e5691b285f354
I20260929 03:00:13.820600 20372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:00:13.836356 20372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583: Bootstrap starting.
I20260929 03:00:13.839823 20372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583: Neither blocks nor log segments found. Creating new log.
I20260929 03:00:13.841068 20372 log.cc:824] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583: Log is configured to *not* fsync() on all Append() calls
Sep 29 03:00:13 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:00:13.844218 20372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583: No bootstrap required, opened a new log
I20260929 03:00:13.854133 20375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:13.820750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41658 (local address 127.8.208.126:40907)
0929 03:00:13.830129 (+ 9379us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:13.830145 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:13.830196 (+ 51us) server_negotiation.cc:407] Connection header received
0929 03:00:13.832532 (+ 2336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:13.832566 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:13.833108 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:13.833613 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:00:13.835116 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:13.837191 (+ 2075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:13.839074 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:13.840171 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:13.840414 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:13.845134 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:00:13.845146 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:00:13.845165 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:00:13.845234 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:00:13.849313 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:13.850140 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:13.850148 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:13.850157 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:13.850334 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:13.850774 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:13.850779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:13.850783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:13.851472 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:00:13.851809 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:13.852283 (+ 474us) server_negotiation.cc:299] Negotiation successful
0929 03:00:13.852725 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9109,"thread_start_us":8883,"threads_started":1}
I20260929 03:00:13.854754 20372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa8d3fb57954c0c88d8d0d5b8ba2583" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40907 } }
I20260929 03:00:13.855201 20372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:00:13.855346 20372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fa8d3fb57954c0c88d8d0d5b8ba2583, State: Initialized, Role: FOLLOWER
I20260929 03:00:13.855934 20372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa8d3fb57954c0c88d8d0d5b8ba2583" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40907 } }
I20260929 03:00:13.856191 20372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:00:13.856367 20372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:00:13.856570 20372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:00:13.859020 20372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa8d3fb57954c0c88d8d0d5b8ba2583" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40907 } }
I20260929 03:00:13.859510 20372 leader_election.cc:302] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fa8d3fb57954c0c88d8d0d5b8ba2583; no voters:
I20260929 03:00:13.860150 20372 leader_election.cc:288] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:00:13.860242 20377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:00:13.863360 20372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:00:13.867599 20377 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [term 1 LEADER]: Becoming Leader. State: Replica: 2fa8d3fb57954c0c88d8d0d5b8ba2583, State: Running, Role: LEADER
I20260929 03:00:13.868368 20377 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa8d3fb57954c0c88d8d0d5b8ba2583" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40907 } }
I20260929 03:00:13.871515 20378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fa8d3fb57954c0c88d8d0d5b8ba2583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa8d3fb57954c0c88d8d0d5b8ba2583" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40907 } } }
I20260929 03:00:13.871912 20378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [sys.catalog]: This master's current role is: LEADER
I20260929 03:00:13.872406 20379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fa8d3fb57954c0c88d8d0d5b8ba2583. Latest consensus state: current_term: 1 leader_uuid: "2fa8d3fb57954c0c88d8d0d5b8ba2583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fa8d3fb57954c0c88d8d0d5b8ba2583" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40907 } } }
I20260929 03:00:13.872704 20379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583 [sys.catalog]: This master's current role is: LEADER
I20260929 03:00:13.880173 20383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:00:13.896382 20383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:00:13.916482 20383 catalog_manager.cc:1347] Generated new cluster ID: f44da416ac6e4686bbec6bc796f418a3
I20260929 03:00:13.916673 20383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:00:13.947423 20372 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 03:00:13.948173 20383 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:00:13.950544 20383 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:00:13.974061 20383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583: Generated new TSK 0
I20260929 03:00:13.975760 20383 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 03:00:14.753681 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.208.84:46189/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:40907
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:00:15 dist-test-slave-m1rv krb5kdc[19861](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790650815, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 03:00:15.164546 20418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:00:15.165001 20418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:00:15.165259 20418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:00:15.177106 20418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:00:15.177376 20418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:00:15.177610 20418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:00:15.194123 20418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.208.84:46189/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:40907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:00:15.196026 20418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:00:15.197870 20418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:00:15.219708 20428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:00:15.221092 20429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:00:15.226919 20418 server_base.cc:1034] running on GCE node
W20260929 03:00:15.227658 20432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:00:15.228367 20418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:00:15.229470 20418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:00:15.230728 20418 hybrid_clock.cc:648] HybridClock initialized: now 1790650815230657 us; error 63 us; skew 500 ppm
I20260929 03:00:15.231164 20418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:00:15 dist-test-slave-m1rv krb5kdc[19861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650815, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:00:15.234472 20418 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 03:00:15.236508 20418 webserver.cc:460] Webserver started at http://127.8.208.65:36387/ using document root <none> and password file <none>
I20260929 03:00:15.237356 20418 fs_manager.cc:359] Metadata directory not provided
I20260929 03:00:15.237497 20418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:00:15.237850 20418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:00:15 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650815, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:00:15.254813 20418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "a2fbbebb8c844d3faea1ec106e054fad"
format_stamp: "Formatted at 2026-09-29 03:00:15 on dist-test-slave-m1rv"
server_key: "ebc2bdd3f683916a3a2a95fbf34bd8af"
server_key_iv: "bbb7b66f8f24654e8e680e331bc85275"
server_key_version: "kuduclusterkey@0"
I20260929 03:00:15.256215 20418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "a2fbbebb8c844d3faea1ec106e054fad"
format_stamp: "Formatted at 2026-09-29 03:00:15 on dist-test-slave-m1rv"
server_key: "ebc2bdd3f683916a3a2a95fbf34bd8af"
server_key_iv: "bbb7b66f8f24654e8e680e331bc85275"
server_key_version: "kuduclusterkey@0"
I20260929 03:00:15.264281 20418 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260929 03:00:15.291240 20438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:00:15.294981 20418 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.005s
I20260929 03:00:15.295202 20418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "a2fbbebb8c844d3faea1ec106e054fad"
format_stamp: "Formatted at 2026-09-29 03:00:15 on dist-test-slave-m1rv"
server_key: "ebc2bdd3f683916a3a2a95fbf34bd8af"
server_key_iv: "bbb7b66f8f24654e8e680e331bc85275"
server_key_version: "kuduclusterkey@0"
I20260929 03:00:15.295449 20418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:00:15.319006 20418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:00:15.323086 20418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:00:15.323398 20418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:00:15.324447 20418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:00:15.325845 20418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:00:15.325914 20418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:00:15.326040 20418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:00:15.326099 20418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:00:15.394495 20418 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:41573
I20260929 03:00:15.394523 20550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:41573 every 8 connection(s)
I20260929 03:00:15.396390 20418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:00:15.402716 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 20418
I20260929 03:00:15.402923 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 03:00:15.403772 9025 external_mini_cluster.cc:1428] Setting key c1e897f9dca9bb401000bfd1d961f285
Sep 29 03:00:15 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650815, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:00:15.461550 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:15.416116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:46543 (local address 127.8.208.126:40907)
0929 03:00:15.441731 (+ 25615us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:15.441738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:15.441760 (+ 22us) server_negotiation.cc:407] Connection header received
0929 03:00:15.441825 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:15.441830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:15.441921 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:15.442060 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:00:15.445117 (+ 3057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:15.446301 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:15.448687 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:15.449403 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:15.449497 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:15.454071 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:00:15.454078 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:00:15.454083 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:00:15.454122 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:00:15.457748 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:15.458820 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:15.458825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:15.458829 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:15.458969 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:15.459527 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:15.459532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:15.459535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:15.459876 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:00:15.459992 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:15.460959 (+ 967us) server_negotiation.cc:299] Negotiation successful
0929 03:00:15.461180 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25474,"thread_start_us":26065,"threads_started":1}
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
I20260929 03:00:15.463716 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:15.415346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:40907 (local address 127.8.208.65:46543)
0929 03:00:15.437143 (+ 21797us) negotiation.cc:107] Waiting for socket to connect
0929 03:00:15.437251 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0929 03:00:15.440029 (+ 2778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:00:15.442628 (+ 2599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:00:15.442642 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:00:15.443393 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:00:15.444290 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:15.444326 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:15.446508 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:15.446514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:15.448522 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:15.448537 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:15.449823 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:15.449831 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:15.450136 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:15.451044 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:00:15.451074 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:00:15.453786 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:00:15.457938 (+ 4152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:00:15.457947 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:00:15.457964 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:00:15.458649 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:00:15.459116 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:00:15.459121 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:00:15.459125 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:00:15.459382 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:00:15.460098 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:00:15.460106 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:00:15.460523 (+ 417us) client_negotiation.cc:769] Sending connection context
0929 03:00:15.461877 (+ 1354us) client_negotiation.cc:240] Negotiation successful
0929 03:00:15.462388 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21368,"thread_start_us":25005,"threads_started":1}
I20260929 03:00:15.465852 20551 heartbeater.cc:347] Connected to a master server at 127.8.208.126:40907
I20260929 03:00:15.466207 20551 heartbeater.cc:464] Registering TS with master...
I20260929 03:00:15.467113 20551 heartbeater.cc:511] Master 127.8.208.126:40907 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
I20260929 03:00:15.469743 20337 ts_manager.cc:194] Registered new tserver with Master: a2fbbebb8c844d3faea1ec106e054fad (127.8.208.65:41573)
I20260929 03:00:15.471861 20337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:46543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 03:00:15.505532 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.208.84:46189/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:40907
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:00:15 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650815, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
Sep 29 03:00:15 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650815, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
W20260929 03:00:15.955678 20557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:00:15.956023 20557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:00:15.956197 20557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:00:15.967208 20557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:00:15.967459 20557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:00:15.967736 20557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:00:15.983166 20557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.208.84:46189/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:40907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:00:15.985139 20557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:00:15.987066 20557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:00:16.021458 20565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:00:16.025388 20564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:00:16.028053 20568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:00:16.028712 20557 server_base.cc:1034] running on GCE node
I20260929 03:00:16.029582 20557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:00:16.038904 20557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:00:16.040407 20557 hybrid_clock.cc:648] HybridClock initialized: now 1790650816040287 us; error 115 us; skew 500 ppm
I20260929 03:00:16.040844 20557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:00:16 dist-test-slave-m1rv krb5kdc[19861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650816, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:00:16.045228 20557 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 03:00:16.073695 20557 webserver.cc:460] Webserver started at http://127.8.208.66:33043/ using document root <none> and password file <none>
I20260929 03:00:16.074826 20557 fs_manager.cc:359] Metadata directory not provided
I20260929 03:00:16.075033 20557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:00:16.075456 20557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:00:16 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650816, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:00:16.095042 20557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "9994688e37e64b50a19fdb1209cfc171"
format_stamp: "Formatted at 2026-09-29 03:00:16 on dist-test-slave-m1rv"
server_key: "89bf98b61bc4b936394cbd2b23c55c64"
server_key_iv: "37351746b001adb1f990c890c9625e32"
server_key_version: "kuduclusterkey@0"
I20260929 03:00:16.096210 20557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "9994688e37e64b50a19fdb1209cfc171"
format_stamp: "Formatted at 2026-09-29 03:00:16 on dist-test-slave-m1rv"
server_key: "89bf98b61bc4b936394cbd2b23c55c64"
server_key_iv: "37351746b001adb1f990c890c9625e32"
server_key_version: "kuduclusterkey@0"
I20260929 03:00:16.103807 20557 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260929 03:00:16.137197 20576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:00:16.139241 20557 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.005s sys 0.001s
I20260929 03:00:16.139508 20557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "9994688e37e64b50a19fdb1209cfc171"
format_stamp: "Formatted at 2026-09-29 03:00:16 on dist-test-slave-m1rv"
server_key: "89bf98b61bc4b936394cbd2b23c55c64"
server_key_iv: "37351746b001adb1f990c890c9625e32"
server_key_version: "kuduclusterkey@0"
I20260929 03:00:16.139837 20557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:00:16.163595 20557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:00:16.167349 20557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:00:16.167619 20557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:00:16.168488 20557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:00:16.169854 20557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:00:16.169926 20557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:00:16.170018 20557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:00:16.170063 20557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:00:16.202056 20557 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:44313
I20260929 03:00:16.202092 20693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:44313 every 8 connection(s)
I20260929 03:00:16.203505 20557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:00:16.213622 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 20557
I20260929 03:00:16.213869 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 03:00:16.214468 9025 external_mini_cluster.cc:1428] Setting key a395b29c31ee931c1366970109ef764e
Sep 29 03:00:16 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650816, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:00:16.223402 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.206345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:57773 (local address 127.8.208.126:40907)
0929 03:00:16.206830 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:16.206835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:16.207753 (+ 918us) server_negotiation.cc:407] Connection header received
0929 03:00:16.209561 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:16.209565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:16.209628 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:16.209720 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:00:16.211064 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.211809 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.213446 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.214073 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.214168 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.217831 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:00:16.217837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:00:16.217842 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:00:16.217881 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:00:16.220423 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:16.221160 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:16.221166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:16.221190 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:16.221313 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:16.221737 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:16.221742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:16.221745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:16.222019 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:00:16.222173 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:16.222838 (+ 665us) server_negotiation.cc:299] Negotiation successful
0929 03:00:16.223045 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":233,"threads_started":1}
I20260929 03:00:16.224352 20697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.206917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:40907 (local address 127.8.208.66:57773)
0929 03:00:16.207591 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0929 03:00:16.207636 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 03:00:16.209327 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:00:16.209912 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:00:16.209929 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:00:16.210366 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:00:16.210926 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.210943 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.211942 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.211946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.213268 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.213318 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.214298 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.214307 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.214591 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.215350 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:00:16.215380 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:00:16.217636 (+ 2256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:00:16.220573 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:00:16.220582 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:00:16.220599 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:00:16.221037 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:00:16.221420 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:00:16.221426 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:00:16.221429 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:00:16.221617 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:00:16.222154 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:00:16.222162 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:00:16.222496 (+ 334us) client_negotiation.cc:769] Sending connection context
0929 03:00:16.222754 (+ 258us) client_negotiation.cc:240] Negotiation successful
0929 03:00:16.223263 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":248,"threads_started":1}
I20260929 03:00:16.225795 20694 heartbeater.cc:347] Connected to a master server at 127.8.208.126:40907
I20260929 03:00:16.226097 20694 heartbeater.cc:464] Registering TS with master...
I20260929 03:00:16.226753 20694 heartbeater.cc:511] Master 127.8.208.126:40907 requested a full tablet report, sending...
I20260929 03:00:16.228188 20337 ts_manager.cc:194] Registered new tserver with Master: 9994688e37e64b50a19fdb1209cfc171 (127.8.208.66:44313)
I20260929 03:00:16.229451 20337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:57773
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 03:00:16.264859 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.208.84:46189/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:40907
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:00:16.479991 20551 heartbeater.cc:503] Master 127.8.208.126:40907 was elected leader, sending a full tablet report...
W20260929 03:00:16.587401 20700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:00:16.587677 20700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:00:16.587867 20700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:00:16.595153 20700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:00:16.595266 20700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:00:16.595440 20700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:00:16.604897 20700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.208.84:46189/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:40907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:00:16.606201 20700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:00:16.607430 20700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:00:16.620587 20705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:00:16.620455 20708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:00:16.620550 20706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:00:16.621286 20700 server_base.cc:1034] running on GCE node
I20260929 03:00:16.621850 20700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:00:16.622716 20700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:00:16.623939 20700 hybrid_clock.cc:648] HybridClock initialized: now 1790650816623892 us; error 54 us; skew 500 ppm
I20260929 03:00:16.624257 20700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:00:16 dist-test-slave-m1rv krb5kdc[19861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650816, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:00:16.627301 20700 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 03:00:16.628628 20700 webserver.cc:460] Webserver started at http://127.8.208.67:46867/ using document root <none> and password file <none>
I20260929 03:00:16.629362 20700 fs_manager.cc:359] Metadata directory not provided
I20260929 03:00:16.629500 20700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:00:16.629803 20700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:00:16 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650816, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:00:16.645919 20700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "47e9894d9f5740e6b2dcc39b4d7446a6"
format_stamp: "Formatted at 2026-09-29 03:00:16 on dist-test-slave-m1rv"
server_key: "5ac51468183ff0a80b6431bba2f45897"
server_key_iv: "0545fb61e5e91f28112a4300e99630c7"
server_key_version: "kuduclusterkey@0"
I20260929 03:00:16.646916 20700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "47e9894d9f5740e6b2dcc39b4d7446a6"
format_stamp: "Formatted at 2026-09-29 03:00:16 on dist-test-slave-m1rv"
server_key: "5ac51468183ff0a80b6431bba2f45897"
server_key_iv: "0545fb61e5e91f28112a4300e99630c7"
server_key_version: "kuduclusterkey@0"
I20260929 03:00:16.652177 20700 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 03:00:16.667618 20715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:00:16.669353 20700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:00:16.669519 20700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "47e9894d9f5740e6b2dcc39b4d7446a6"
format_stamp: "Formatted at 2026-09-29 03:00:16 on dist-test-slave-m1rv"
server_key: "5ac51468183ff0a80b6431bba2f45897"
server_key_iv: "0545fb61e5e91f28112a4300e99630c7"
server_key_version: "kuduclusterkey@0"
I20260929 03:00:16.669775 20700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:00:16.694283 20700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:00:16.698518 20700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:00:16.698766 20700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:00:16.699779 20700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:00:16.701303 20700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:00:16.701375 20700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:00:16.701470 20700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:00:16.701516 20700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:00:16.734053 20700 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:38949
I20260929 03:00:16.734095 20827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:38949 every 8 connection(s)
I20260929 03:00:16.735335 20700 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:00:16.737350 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 20700
I20260929 03:00:16.737547 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 03:00:16.738104 9025 external_mini_cluster.cc:1428] Setting key 70ef3e423215da82214e1b9188de72bd
Sep 29 03:00:16 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650816, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:00:16.755142 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.737946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:45437 (local address 127.8.208.126:40907)
0929 03:00:16.738424 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:16.738430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:16.739567 (+ 1137us) server_negotiation.cc:407] Connection header received
0929 03:00:16.741808 (+ 2241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:16.741813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:16.741905 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:16.742031 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:00:16.743418 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.744259 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.745864 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.746282 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.746411 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.749485 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:00:16.749490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:00:16.749493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:00:16.749523 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:00:16.752197 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:16.753078 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:16.753086 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:16.753091 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:16.753274 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:16.753768 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:16.753771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:16.753774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:16.754049 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:00:16.754158 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:16.754721 (+ 563us) server_negotiation.cc:299] Negotiation successful
0929 03:00:16.754876 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":225,"threads_started":1}
I20260929 03:00:16.756045 20831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.738563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:40907 (local address 127.8.208.67:45437)
0929 03:00:16.739406 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0929 03:00:16.739472 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 03:00:16.741571 (+ 2099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:00:16.742252 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:00:16.742259 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:00:16.742674 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:00:16.743267 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.743286 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.744400 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.744407 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.745723 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.745733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.746501 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.746509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.746722 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.747295 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:00:16.747318 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:00:16.749287 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:00:16.752329 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:00:16.752339 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:00:16.752352 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:00:16.752936 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:00:16.753387 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:00:16.753391 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:00:16.753394 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:00:16.753644 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:00:16.754153 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:00:16.754158 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:00:16.754430 (+ 272us) client_negotiation.cc:769] Sending connection context
0929 03:00:16.754642 (+ 212us) client_negotiation.cc:240] Negotiation successful
0929 03:00:16.754983 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":344,"threads_started":1}
I20260929 03:00:16.757364 20828 heartbeater.cc:347] Connected to a master server at 127.8.208.126:40907
I20260929 03:00:16.757653 20828 heartbeater.cc:464] Registering TS with master...
I20260929 03:00:16.758391 20828 heartbeater.cc:511] Master 127.8.208.126:40907 requested a full tablet report, sending...
I20260929 03:00:16.759687 20337 ts_manager.cc:194] Registered new tserver with Master: 47e9894d9f5740e6b2dcc39b4d7446a6 (127.8.208.67:38949)
I20260929 03:00:16.760758 20337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:45437
I20260929 03:00:16.766353 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:00:16.783973 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.771730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41660 (local address 127.8.208.126:40907)
0929 03:00:16.771982 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:16.771991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:16.772441 (+ 450us) server_negotiation.cc:407] Connection header received
0929 03:00:16.772572 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:16.772576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:16.772646 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:16.772726 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:00:16.774124 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.774845 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.776291 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.776728 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.776831 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.779110 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:00:16.779114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:00:16.779117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:00:16.779149 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:00:16.781709 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:16.782322 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:16.782326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:16.782329 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:16.782456 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:16.782775 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:16.782778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:16.782780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:16.783032 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:00:16.783206 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:16.783469 (+ 263us) server_negotiation.cc:299] Negotiation successful
0929 03:00:16.783633 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260929 03:00:16.787856 20337 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:00:16.794921 20337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:00:16.817544 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.807879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:41573 (local address 127.0.0.1:38954)
0929 03:00:16.808461 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0929 03:00:16.808483 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 03:00:16.808622 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:00:16.809799 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:00:16.809804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:00:16.809832 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:00:16.810241 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.810253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.812336 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.812342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.814498 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.814508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.816417 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.816424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.816639 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.816705 (+ 66us) client_negotiation.cc:769] Sending connection context
0929 03:00:16.816845 (+ 140us) client_negotiation.cc:240] Negotiation successful
0929 03:00:16.817027 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":85,"thread_start_us":288,"threads_started":1}
I20260929 03:00:16.817569 20841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.806482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:44313 (local address 127.0.0.1:37050)
0929 03:00:16.807561 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0929 03:00:16.807599 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 03:00:16.807811 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:00:16.808623 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:00:16.808631 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:00:16.808669 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:00:16.809098 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.809112 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.811364 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.811369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.813974 (+ 2605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.813989 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.816750 (+ 2761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.816757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.816911 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.816940 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 03:00:16.817030 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 03:00:16.817146 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"mutex_wait_us":24,"thread_start_us":777,"threads_started":1}
I20260929 03:00:16.818252 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.806651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37050 (local address 127.8.208.66:44313)
0929 03:00:16.807226 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:16.807236 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:16.807711 (+ 475us) server_negotiation.cc:407] Connection header received
0929 03:00:16.808093 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:16.808105 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:16.808369 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:16.809330 (+ 961us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:00:16.809578 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.811214 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.814736 (+ 3522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.816162 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.816417 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.816645 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:16.817711 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0929 03:00:16.817869 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":307,"threads_started":1}
I20260929 03:00:16.819190 20844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.808152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38954 (local address 127.8.208.65:41573)
0929 03:00:16.809217 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:16.809227 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:16.809257 (+ 30us) server_negotiation.cc:407] Connection header received
0929 03:00:16.809350 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:16.809356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:16.809579 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:16.810447 (+ 868us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:00:16.810666 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.812168 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.814643 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.816162 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.818571 (+ 2409us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.818775 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:16.818831 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 03:00:16.818945 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":882,"thread_start_us":732,"threads_started":1}
I20260929 03:00:16.825623 20629 tablet_service.cc:1467] Processing CreateTablet for tablet 0bbc13fd36964a46a99752720a870e1e (DEFAULT_TABLE table=test-table [id=2dfd6fc58b3e49a2a5cee478bd56eef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:00:16.826108 20486 tablet_service.cc:1467] Processing CreateTablet for tablet 0bbc13fd36964a46a99752720a870e1e (DEFAULT_TABLE table=test-table [id=2dfd6fc58b3e49a2a5cee478bd56eef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:00:16.827272 20629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bbc13fd36964a46a99752720a870e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:00:16.827471 20486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bbc13fd36964a46a99752720a870e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:00:16.832304 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.808469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:38949 (local address 127.0.0.1:42580)
0929 03:00:16.818036 (+ 9567us) negotiation.cc:107] Waiting for socket to connect
0929 03:00:16.818058 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 03:00:16.818186 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:00:16.825012 (+ 6826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:00:16.825017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:00:16.825044 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:00:16.825272 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.825283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.827357 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.827364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.829439 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.829467 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.831429 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.831436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.831658 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.831714 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 03:00:16.831809 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 03:00:16.831966 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9472,"thread_start_us":12913,"threads_started":1}
I20260929 03:00:16.833472 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.809028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42580 (local address 127.8.208.67:38949)
0929 03:00:16.824305 (+ 15277us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:16.824314 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:16.824344 (+ 30us) server_negotiation.cc:407] Connection header received
0929 03:00:16.824427 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:16.824434 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:16.824653 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:16.825438 (+ 785us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:00:16.825683 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.827202 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.829679 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.831232 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.832518 (+ 1286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.832742 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:16.832896 (+ 154us) server_negotiation.cc:299] Negotiation successful
0929 03:00:16.833053 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15113,"thread_start_us":15724,"threads_started":1}
I20260929 03:00:16.841312 20763 tablet_service.cc:1467] Processing CreateTablet for tablet 0bbc13fd36964a46a99752720a870e1e (DEFAULT_TABLE table=test-table [id=2dfd6fc58b3e49a2a5cee478bd56eef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:00:16.842759 20848 tablet_bootstrap.cc:492] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad: Bootstrap starting.
I20260929 03:00:16.843014 20763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bbc13fd36964a46a99752720a870e1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:00:16.845013 20847 tablet_bootstrap.cc:492] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171: Bootstrap starting.
I20260929 03:00:16.847266 20848 tablet_bootstrap.cc:654] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad: Neither blocks nor log segments found. Creating new log.
I20260929 03:00:16.848670 20848 log.cc:824] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad: Log is configured to *not* fsync() on all Append() calls
I20260929 03:00:16.848762 20847 tablet_bootstrap.cc:654] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171: Neither blocks nor log segments found. Creating new log.
I20260929 03:00:16.849997 20847 log.cc:824] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171: Log is configured to *not* fsync() on all Append() calls
I20260929 03:00:16.852113 20848 tablet_bootstrap.cc:492] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad: No bootstrap required, opened a new log
I20260929 03:00:16.852404 20848 ts_tablet_manager.cc:1397] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260929 03:00:16.853094 20847 tablet_bootstrap.cc:492] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171: No bootstrap required, opened a new log
I20260929 03:00:16.853466 20847 ts_tablet_manager.cc:1397] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260929 03:00:16.856294 20851 tablet_bootstrap.cc:492] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6: Bootstrap starting.
I20260929 03:00:16.859426 20851 tablet_bootstrap.cc:654] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6: Neither blocks nor log segments found. Creating new log.
I20260929 03:00:16.860707 20851 log.cc:824] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6: Log is configured to *not* fsync() on all Append() calls
I20260929 03:00:16.863046 20851 tablet_bootstrap.cc:492] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6: No bootstrap required, opened a new log
I20260929 03:00:16.863337 20851 ts_tablet_manager.cc:1397] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260929 03:00:16.863809 20848 raft_consensus.cc:348] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 } } peers { permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 } } peers { permanent_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 38949 } }
I20260929 03:00:16.864333 20848 raft_consensus.cc:374] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:00:16.864459 20848 raft_consensus.cc:729] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fbbebb8c844d3faea1ec106e054fad, State: Initialized, Role: FOLLOWER
I20260929 03:00:16.864351 20847 raft_consensus.cc:348] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 } } peers { permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 } } peers { permanent_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 38949 } }
I20260929 03:00:16.864768 20847 raft_consensus.cc:374] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:00:16.864886 20847 raft_consensus.cc:729] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9994688e37e64b50a19fdb1209cfc171, State: Initialized, Role: FOLLOWER
I20260929 03:00:16.865226 20848 consensus_queue.cc:260] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 } } peers { permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 } } peers { permanent_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 38949 } }
I20260929 03:00:16.865491 20847 consensus_queue.cc:260] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 } } peers { permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 } } peers { permanent_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 38949 } }
I20260929 03:00:16.868970 20848 ts_tablet_manager.cc:1428] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260929 03:00:16.870029 20694 heartbeater.cc:503] Master 127.8.208.126:40907 was elected leader, sending a full tablet report...
I20260929 03:00:16.870613 20847 ts_tablet_manager.cc:1428] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20260929 03:00:16.874424 20851 raft_consensus.cc:348] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 } } peers { permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 } } peers { permanent_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 38949 } }
I20260929 03:00:16.874900 20851 raft_consensus.cc:374] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:00:16.875061 20851 raft_consensus.cc:729] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47e9894d9f5740e6b2dcc39b4d7446a6, State: Initialized, Role: FOLLOWER
I20260929 03:00:16.875662 20851 consensus_queue.cc:260] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 } } peers { permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 } } peers { permanent_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 38949 } }
I20260929 03:00:16.876583 20828 heartbeater.cc:503] Master 127.8.208.126:40907 was elected leader, sending a full tablet report...
I20260929 03:00:16.877218 20851 ts_tablet_manager.cc:1428] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260929 03:00:16.880621 20855 raft_consensus.cc:482] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:00:16.880858 20855 raft_consensus.cc:504] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 } } peers { permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 } } peers { permanent_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 38949 } }
I20260929 03:00:16.881701 20853 raft_consensus.cc:482] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:00:16.881932 20853 raft_consensus.cc:504] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 } } peers { permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 } } peers { permanent_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 38949 } }
I20260929 03:00:16.882513 20855 leader_election.cc:288] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9994688e37e64b50a19fdb1209cfc171 (127.8.208.66:44313), a2fbbebb8c844d3faea1ec106e054fad (127.8.208.65:41573)
I20260929 03:00:16.883698 20853 leader_election.cc:288] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9994688e37e64b50a19fdb1209cfc171 (127.8.208.66:44313), 47e9894d9f5740e6b2dcc39b4d7446a6 (127.8.208.67:38949)
I20260929 03:00:16.892087 20831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.882868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:44313 (local address 127.8.208.67:34829)
0929 03:00:16.883045 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 03:00:16.883062 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:00:16.883150 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:00:16.884335 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:00:16.884340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:00:16.884365 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:00:16.884562 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.884577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.886472 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.886479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.889026 (+ 2547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.889044 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.891219 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.891224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.891443 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.891501 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 03:00:16.891601 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 03:00:16.891762 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":1}
I20260929 03:00:16.893002 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.883682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:34829 (local address 127.8.208.66:44313)
0929 03:00:16.883890 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:16.883897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:16.883918 (+ 21us) server_negotiation.cc:407] Connection header received
0929 03:00:16.883988 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:16.883996 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:16.884080 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:16.884171 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:00:16.884735 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.886071 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.889240 (+ 3169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.891061 (+ 1821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.892503 (+ 1442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.892574 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:16.892623 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 03:00:16.892744 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 03:00:16.896485 20649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bbc13fd36964a46a99752720a870e1e" candidate_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9994688e37e64b50a19fdb1209cfc171" is_pre_election: true
I20260929 03:00:16.897089 20649 raft_consensus.cc:2454] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47e9894d9f5740e6b2dcc39b4d7446a6 in term 0.
I20260929 03:00:16.897833 20716 leader_election.cc:302] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47e9894d9f5740e6b2dcc39b4d7446a6, 9994688e37e64b50a19fdb1209cfc171; no voters:
I20260929 03:00:16.898903 20844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.883517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:36107 (local address 127.8.208.65:41573)
0929 03:00:16.884229 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:16.884234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:16.890709 (+ 6475us) server_negotiation.cc:407] Connection header received
0929 03:00:16.890872 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:16.890877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:16.890960 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:16.891057 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:00:16.891893 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.893021 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.896081 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.897529 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.897736 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.897888 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:16.898529 (+ 641us) server_negotiation.cc:299] Negotiation successful
0929 03:00:16.898653 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":562}
I20260929 03:00:16.899518 20855 raft_consensus.cc:2781] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:00:16.899715 20855 raft_consensus.cc:482] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:00:16.899897 20855 raft_consensus.cc:3037] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:00:16.900375 20856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.883263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:41573 (local address 127.8.208.67:36107)
0929 03:00:16.890580 (+ 7317us) negotiation.cc:107] Waiting for socket to connect
0929 03:00:16.890602 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 03:00:16.890742 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:00:16.891223 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:00:16.891227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:00:16.891249 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:00:16.891704 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.891715 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.893242 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.893247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.895488 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.895501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.898155 (+ 2654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.898161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.898333 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.898368 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 03:00:16.899205 (+ 837us) client_negotiation.cc:240] Negotiation successful
0929 03:00:16.899699 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7161,"mutex_wait_us":355,"thread_start_us":6979,"threads_started":1}
I20260929 03:00:16.901541 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.890487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:57395 (local address 127.8.208.67:38949)
0929 03:00:16.890632 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:16.890636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:16.891435 (+ 799us) server_negotiation.cc:407] Connection header received
0929 03:00:16.891724 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:16.891731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:16.891872 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:16.891945 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:00:16.892942 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.894440 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.898903 (+ 4463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.900369 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.900533 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.900598 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:16.901161 (+ 563us) server_negotiation.cc:299] Negotiation successful
0929 03:00:16.901308 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":33}
I20260929 03:00:16.901485 20506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bbc13fd36964a46a99752720a870e1e" candidate_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fbbebb8c844d3faea1ec106e054fad" is_pre_election: true
I20260929 03:00:16.901929 20506 raft_consensus.cc:2454] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47e9894d9f5740e6b2dcc39b4d7446a6 in term 0.
I20260929 03:00:16.902706 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.883850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:44313 (local address 127.8.208.65:34235)
0929 03:00:16.884569 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0929 03:00:16.884598 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 03:00:16.884758 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:00:16.893816 (+ 9058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:00:16.893824 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:00:16.893847 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:00:16.894213 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.894227 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.895665 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.895671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.899377 (+ 3706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.899391 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.901944 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.901960 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.902137 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.902167 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 03:00:16.902262 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 03:00:16.902402 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":713,"threads_started":1}
I20260929 03:00:16.903247 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.884984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:38949 (local address 127.8.208.65:57395)
0929 03:00:16.891327 (+ 6343us) negotiation.cc:107] Waiting for socket to connect
0929 03:00:16.891347 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:00:16.891545 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:00:16.892175 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:00:16.892183 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:00:16.892228 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:00:16.892724 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.892739 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.894746 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.894755 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.898682 (+ 3927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:00:16.898700 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.900766 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.900771 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:00:16.900971 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.901004 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 03:00:16.901698 (+ 694us) client_negotiation.cc:240] Negotiation successful
0929 03:00:16.901854 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6192,"mutex_wait_us":1,"thread_start_us":6026,"threads_started":1}
I20260929 03:00:16.903301 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.884423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:34235 (local address 127.8.208.66:44313)
0929 03:00:16.893417 (+ 8994us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:16.893425 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:16.893443 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:00:16.893500 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:16.893504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:16.893571 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:16.893657 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:00:16.894382 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.895466 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.899557 (+ 4091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.900794 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.900947 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.901013 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:16.902891 (+ 1878us) server_negotiation.cc:299] Negotiation successful
0929 03:00:16.903024 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8889,"thread_start_us":15482,"threads_started":1}
I20260929 03:00:16.903642 20855 raft_consensus.cc:504] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 } } peers { permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 } } peers { permanent_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 38949 } }
I20260929 03:00:16.903867 20649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bbc13fd36964a46a99752720a870e1e" candidate_uuid: "a2fbbebb8c844d3faea1ec106e054fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9994688e37e64b50a19fdb1209cfc171" is_pre_election: true
I20260929 03:00:16.904109 20649 raft_consensus.cc:2454] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fbbebb8c844d3faea1ec106e054fad in term 0.
I20260929 03:00:16.904393 20855 leader_election.cc:288] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [CANDIDATE]: Term 1 election: Requested vote from peers 9994688e37e64b50a19fdb1209cfc171 (127.8.208.66:44313), a2fbbebb8c844d3faea1ec106e054fad (127.8.208.65:41573)
I20260929 03:00:16.904388 20783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bbc13fd36964a46a99752720a870e1e" candidate_uuid: "a2fbbebb8c844d3faea1ec106e054fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" is_pre_election: true
I20260929 03:00:16.904552 20439 leader_election.cc:302] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9994688e37e64b50a19fdb1209cfc171, a2fbbebb8c844d3faea1ec106e054fad; no voters:
I20260929 03:00:16.904667 20649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bbc13fd36964a46a99752720a870e1e" candidate_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9994688e37e64b50a19fdb1209cfc171"
I20260929 03:00:16.904896 20783 raft_consensus.cc:2379] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fbbebb8c844d3faea1ec106e054fad in current term 1: Already voted for candidate 47e9894d9f5740e6b2dcc39b4d7446a6 in this term.
I20260929 03:00:16.904908 20649 raft_consensus.cc:3037] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:00:16.904989 20853 raft_consensus.cc:2781] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:00:16.905123 20853 raft_consensus.cc:482] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:00:16.905239 20853 raft_consensus.cc:3037] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:00:16.907279 20649 raft_consensus.cc:2454] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47e9894d9f5740e6b2dcc39b4d7446a6 in term 1.
I20260929 03:00:16.907294 20853 raft_consensus.cc:504] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 } } peers { permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 } } peers { permanent_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 38949 } }
I20260929 03:00:16.907742 20716 leader_election.cc:302] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47e9894d9f5740e6b2dcc39b4d7446a6, 9994688e37e64b50a19fdb1209cfc171; no voters:
I20260929 03:00:16.907910 20853 leader_election.cc:288] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [CANDIDATE]: Term 1 election: Requested vote from peers 9994688e37e64b50a19fdb1209cfc171 (127.8.208.66:44313), 47e9894d9f5740e6b2dcc39b4d7446a6 (127.8.208.67:38949)
I20260929 03:00:16.908027 20506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bbc13fd36964a46a99752720a870e1e" candidate_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fbbebb8c844d3faea1ec106e054fad"
I20260929 03:00:16.908301 20506 raft_consensus.cc:2379] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 47e9894d9f5740e6b2dcc39b4d7446a6 in current term 1: Already voted for candidate a2fbbebb8c844d3faea1ec106e054fad in this term.
I20260929 03:00:16.908402 20855 raft_consensus.cc:2781] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:00:16.908428 20649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bbc13fd36964a46a99752720a870e1e" candidate_uuid: "a2fbbebb8c844d3faea1ec106e054fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9994688e37e64b50a19fdb1209cfc171"
I20260929 03:00:16.908689 20649 raft_consensus.cc:2379] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fbbebb8c844d3faea1ec106e054fad in current term 1: Already voted for candidate 47e9894d9f5740e6b2dcc39b4d7446a6 in this term.
I20260929 03:00:16.909467 20855 raft_consensus.cc:686] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 1 LEADER]: Becoming Leader. State: Replica: 47e9894d9f5740e6b2dcc39b4d7446a6, State: Running, Role: LEADER
I20260929 03:00:16.909467 20783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bbc13fd36964a46a99752720a870e1e" candidate_uuid: "a2fbbebb8c844d3faea1ec106e054fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6"
I20260929 03:00:16.909982 20439 leader_election.cc:302] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fbbebb8c844d3faea1ec106e054fad; no voters: 47e9894d9f5740e6b2dcc39b4d7446a6, 9994688e37e64b50a19fdb1209cfc171
I20260929 03:00:16.910024 20855 consensus_queue.cc:237] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 } } peers { permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 } } peers { permanent_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 38949 } }
I20260929 03:00:16.910229 20853 raft_consensus.cc:2726] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260929 03:00:16.914311 20336 catalog_manager.cc:5495] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47e9894d9f5740e6b2dcc39b4d7446a6 (127.8.208.67). New cstate: current_term: 1 leader_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47e9894d9f5740e6b2dcc39b4d7446a6" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 38949 } health_report { overall_health: HEALTHY } } }
I20260929 03:00:16.930343 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.924856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42588 (local address 127.8.208.67:38949)
0929 03:00:16.925057 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:16.925062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:16.925080 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:00:16.925294 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:16.925298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:16.925379 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:16.925526 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:00:16.925920 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.926662 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.928067 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.928506 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.928636 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:16.929213 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:00:16.929720 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:00:16.929874 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:16.929969 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 03:00:16.930043 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 03:00:16.938375 20506 raft_consensus.cc:1261] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 1 FOLLOWER]: Refusing update from remote peer 47e9894d9f5740e6b2dcc39b4d7446a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:00:16.938391 20649 raft_consensus.cc:1261] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171 [term 1 FOLLOWER]: Refusing update from remote peer 47e9894d9f5740e6b2dcc39b4d7446a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:00:16.939185 20855 consensus_queue.cc:1035] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fbbebb8c844d3faea1ec106e054fad" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:00:16.939522 20860 consensus_queue.cc:1035] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9994688e37e64b50a19fdb1209cfc171" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 44313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 03:00:16.946359 20552 tablet.cc:2369] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:00:16.953354 20865 mvcc.cc:205] Tried to move back new op lower bound from 7334505746169417728 to 7334505746067628032. Current Snapshot: MvccSnapshot[applied={T|T < 7334505746169417728}]
I20260929 03:00:16.965678 20864 mvcc.cc:205] Tried to move back new op lower bound from 7334505746169417728 to 7334505746067628032. Current Snapshot: MvccSnapshot[applied={T|T < 7334505746169417728 or (T in {7334505746169417728})}]
I20260929 03:00:16.969588 20337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:00:16.969904 20337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:00:16.973627 20337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2fa8d3fb57954c0c88d8d0d5b8ba2583: Sending DeleteTablet for 3 replicas of tablet 0bbc13fd36964a46a99752720a870e1e
I20260929 03:00:16.974700 20629 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bbc13fd36964a46a99752720a870e1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:00:16 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:37050
I20260929 03:00:16.975020 20486 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bbc13fd36964a46a99752720a870e1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:00:16 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:38954
I20260929 03:00:16.975260 20763 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bbc13fd36964a46a99752720a870e1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:00:16 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:42580
I20260929 03:00:16.978153 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 20418
I20260929 03:00:16.984823 20874 tablet_replica.cc:330] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6: stopping tablet replica
I20260929 03:00:16.986564 20874 raft_consensus.cc:2229] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:00:16.989149 20874 raft_consensus.cc:2258] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:00:16.990067 20873 tablet_replica.cc:330] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad: stopping tablet replica
I20260929 03:00:16.990494 20873 raft_consensus.cc:2229] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:00:16.990994 20873 raft_consensus.cc:2258] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:00:16.994122 20874 ts_tablet_manager.cc:1905] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:00:16.996138 20872 tablet_replica.cc:330] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171: stopping tablet replica
I20260929 03:00:16.996564 20872 raft_consensus.cc:2229] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:00:16.998289 20873 ts_tablet_manager.cc:1905] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:00:17.003000 20874 ts_tablet_manager.cc:1918] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:00:17.003211 20874 log.cc:1196] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/0bbc13fd36964a46a99752720a870e1e
I20260929 03:00:17.003448 20873 ts_tablet_manager.cc:1918] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:00:17.003789 20874 ts_tablet_manager.cc:1939] T 0bbc13fd36964a46a99752720a870e1e P 47e9894d9f5740e6b2dcc39b4d7446a6: Deleting consensus metadata
I20260929 03:00:17.004082 20872 raft_consensus.cc:2258] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:00:17.004594 20873 log.cc:1196] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/0bbc13fd36964a46a99752720a870e1e
I20260929 03:00:17.005870 20321 catalog_manager.cc:4841] TS 47e9894d9f5740e6b2dcc39b4d7446a6 (127.8.208.67:38949): tablet 0bbc13fd36964a46a99752720a870e1e (table test-table [id=2dfd6fc58b3e49a2a5cee478bd56eef5]) successfully deleted
I20260929 03:00:17.006242 20872 ts_tablet_manager.cc:1905] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:00:17.006708 20873 ts_tablet_manager.cc:1939] T 0bbc13fd36964a46a99752720a870e1e P a2fbbebb8c844d3faea1ec106e054fad: Deleting consensus metadata
Sep 29 03:00:17 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 03:00:17.008653 20321 catalog_manager.cc:4841] TS a2fbbebb8c844d3faea1ec106e054fad (127.8.208.65:41573): tablet 0bbc13fd36964a46a99752720a870e1e (table test-table [id=2dfd6fc58b3e49a2a5cee478bd56eef5]) successfully deleted
I20260929 03:00:17.013330 20872 ts_tablet_manager.cc:1918] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:00:17.013494 20872 log.cc:1196] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/0bbc13fd36964a46a99752720a870e1e
I20260929 03:00:17.013962 20872 ts_tablet_manager.cc:1939] T 0bbc13fd36964a46a99752720a870e1e P 9994688e37e64b50a19fdb1209cfc171: Deleting consensus metadata
I20260929 03:00:17.015751 20321 catalog_manager.cc:4841] TS 9994688e37e64b50a19fdb1209cfc171 (127.8.208.66:44313): tablet 0bbc13fd36964a46a99752720a870e1e (table test-table [id=2dfd6fc58b3e49a2a5cee478bd56eef5]) successfully deleted
I20260929 03:00:17.016424 20844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:16.980762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38970 (local address 127.8.208.65:41573)
0929 03:00:16.981638 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:16.981652 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:16.985413 (+ 3761us) server_negotiation.cc:407] Connection header received
0929 03:00:16.985819 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:16.985828 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:16.985923 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:16.986007 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:00:16.991459 (+ 5452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.992556 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.995658 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:16.996351 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:16.996527 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:17.008897 (+ 12370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:00:17.008911 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:00:17.008928 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:00:17.008969 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:00:17.012874 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:17.013894 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:17.013904 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:17.013910 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:17.014068 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:17.014559 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:17.014565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:17.014568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:17.015038 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:00:17.015322 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:17.015993 (+ 671us) server_negotiation.cc:299] Negotiation successful
0929 03:00:17.016168 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613}
I20260929 03:00:17.017712 20546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:00:17.134501 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 20418
I20260929 03:00:17.149134 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 20557
Sep 29 03:00:17 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 03:00:17.165828 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:17.150037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37066 (local address 127.8.208.66:44313)
0929 03:00:17.150245 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:17.150251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:17.150273 (+ 22us) server_negotiation.cc:407] Connection header received
0929 03:00:17.150391 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:17.150397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:17.150489 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:17.150599 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:00:17.151832 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:17.152920 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:17.154469 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:17.155105 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:17.155234 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:17.159755 (+ 4521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:00:17.159768 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:00:17.159784 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:00:17.159816 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:00:17.162870 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:17.163818 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:17.163826 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:17.163828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:17.163978 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:17.164369 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:17.164372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:17.164374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:17.164714 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:00:17.165011 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:17.165331 (+ 320us) server_negotiation.cc:299] Negotiation successful
0929 03:00:17.165504 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 03:00:17.166528 20689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:00:17.255566 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 20557
I20260929 03:00:17.272646 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 20700
Sep 29 03:00:17 dist-test-slave-m1rv krb5kdc[19861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 03:00:17.290690 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:00:17.273495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.8.208.67:38949)
0929 03:00:17.273715 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 03:00:17.273720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:00:17.273801 (+ 81us) server_negotiation.cc:407] Connection header received
0929 03:00:17.274038 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:00:17.274042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:00:17.274101 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:00:17.274180 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:00:17.276060 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:17.276808 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:17.278742 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:00:17.279157 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:00:17.279245 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:00:17.284447 (+ 5202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:00:17.284461 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:00:17.284475 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:00:17.284513 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:00:17.287961 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:17.288697 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:17.288704 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:17.288706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:17.288830 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:00:17.289219 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:00:17.289223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:00:17.289226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:00:17.289596 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:00:17.289934 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0929 03:00:17.290136 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 03:00:17.290349 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 03:00:17.291415 20823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:00:17.412154 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 20700
I20260929 03:00:17.431588 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 20305
I20260929 03:00:17.432833 20367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:00:17.545583 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 20305
I20260929 03:00:17.569026 9025 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 03:00:17.904762 9025 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 03:00:17.905009 9025 mini_ranger.cc:66] Stopping Ranger...
I20260929 03:00:18.250780 9025 mini_ranger.cc:68] Stopped Ranger
2026-09-29 03:00:18.251 UTC [19891] LOG: received smart shutdown request
2026-09-29 03:00:18.254 UTC [19891] LOG: background worker "logical replication launcher" (PID 19900) exited with exit code 1
2026-09-29 03:00:18.254 UTC [19895] LOG: shutting down
2026-09-29 03:00:18.431 UTC [19891] LOG: database system is shut down
2026-09-29T03:00:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49344 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:00:18 dist-test-slave-m1rv krb5kdc[20888](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:00:18 dist-test-slave-m1rv krb5kdc[20888](info): set up 2 sockets
Sep 29 03:00:18 dist-test-slave-m1rv krb5kdc[20888](info): commencing operation
krb5kdc: starting...
W20260929 03:00:19.669633 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:00:19 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:00:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:00:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.208.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.208.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/rangeradmin_127.8.208.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/rangeradmin_127.8.208.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.208.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.208.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/rangerlookup_127.8.208.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/rangerlookup_127.8.208.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.208.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.208.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/HTTP_127.8.208.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/HTTP_127.8.208.84@KRBTEST.COM.keytab.
I20260929 03:00:19.988358 9025 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1CUkeB/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/postgres -l logfile start
2026-09-29 03:00:22.582 UTC [20918] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 03:00:22.582 UTC [20918] LOG: listening on IPv4 address "127.8.208.84", port 59407
2026-09-29 03:00:22.586 UTC [20918] LOG: listening on Unix socket "/tmp/.s.PGSQL.59407"
2026-09-29 03:00:22.608 UTC [20921] LOG: database system was shut down at 2026-09-29 03:00:21 UTC
2026-09-29 03:00:22.611 UTC [20918] LOG: database system is ready to accept connections
I20260929 03:00:23.597321 9025 mini_postgres.cc:96] Postgres bound to 59407
2026-09-29 03:00:23.614 UTC [20929] FATAL: database "slave" does not exist
127.8.208.84:59407 - accepting connections
I20260929 03:00:23.614697 9025 mini_ranger.cc:161] Starting Ranger...
I20260929 03:00:23.640064 9025 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 03:00:24.166750 9025 mini_ranger.cc:87] Created ranger Postgres database
I20260929 03:00:24.166882 9025 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-admin
2026-09-29 03:00:24,448 [I] DB FLAVOR :POSTGRES
2026-09-29 03:00:24,448 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 03:00:24,448 [I] Checking connection..
2026-09-29 03:00:24,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 03:00:24,700 [I] Checking connection passed.
2026-09-29 03:00:24,701 [I] --------- Verifying version history table ---------
2026-09-29 03:00:24,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 03:00:24,962 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 03:00:24,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 03:00:25,207 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 03:00:25,207 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 03:00:25,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 03:00:25.429 UTC [20993] WARNING: there is no transaction in progress
2026-09-29 03:00:25,435 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 03:00:25,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 03:00:25,685 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 03:00:25,685 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 03:00:25,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 03:00:25,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 03:00:26,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-m1rv.c.gcp-upstream.internal','N') ;"
2026-09-29 03:00:26,429 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 03:00:26,687 [I] Table x_portal_user does not exist in database ranger
2026-09-29 03:00:26,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 03:00:26,936 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 03:00:26,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 03:00:27,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 03:00:27,428 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 03:00:27,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 03:00:28.361 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.377 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.393 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.408 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.423 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.441 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.505 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.511 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.525 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.536 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.545 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.558 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.568 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.574 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.580 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:28.586 UTC [21101] WARNING: there is no transaction in progress
2026-09-29 03:00:29,218 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 03:00:29,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 03:00:29,440 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 03:00:29,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 03:00:29,686 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 03:00:29,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 03:00:29,918 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 03:00:29,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 03:00:30,159 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 03:00:30,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-m1rv.c.gcp-upstream.internal';"
2026-09-29 03:00:30,663 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 03:00:30,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 03:00:30,923 [I] DB_PATCHES have already been applied
I20260929 03:00:30.929435 9025 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-admin:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1CUkeB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 03:00:30.929531 9025 mini_ranger.cc:193] Using host: 127.8.208.84
I20260929 03:00:30.943421 9025 mini_ranger.cc:227] Ranger admin URL: http://127.8.208.84:33795
Sep 29, 2026 3:00:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 3:00:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 3:00:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 3:00:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 3:00:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 3:00:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 3:00:31 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 03:00:32.083060 9025 mini_ranger.cc:160] Time spent starting Ranger: real 8.468s user 0.002s sys 0.035s
Sep 29, 2026 3:00:51 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 3:00:51 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 3:00:51 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 3:00:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 3:00:51 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 3:00:52 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 03:00:53.484732 9025 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.208.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.208.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/rangerkms_127.8.208.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.208.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/rangerkms_127.8.208.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.208.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.208.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.208.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/HTTP_127.8.208.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.208.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/HTTP_127.8.208.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 03:00:53 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650853, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 03:00:53.662074 9025 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 03:00:53.681823 9025 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 03:00:54.611901 9025 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 03:00:54.612013 9025 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-kms
I20260929 03:00:54.612062 9025 mini_ranger_kms.cc:227] Using postgres at 127.8.208.84:59407
2026-09-29 03:00:54,963 [I] DB FLAVOR :POSTGRES
2026-09-29 03:00:54,963 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 03:00:54,963 [I] Checking connection
2026-09-29 03:00:54,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 03:00:55,198 [I] connection success
2026-09-29 03:00:55,198 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 03:00:55,198 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 03:00:55,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 03:00:55,421 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 03:00:55,421 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 03:00:55,421 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 03:00:55,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.208.84:59407/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 03:00:55,713 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 03:00:56.165182 9025 mini_ranger_kms.cc:310] Created kms service
I20260929 03:00:56.355295 9025 mini_ranger_kms.cc:326] Created kudu user
I20260929 03:00:56.398922 9025 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 03:00:56 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650853, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:00:56.801584 9025 mini_ranger_kms.cc:384] Added ranger policy
I20260929 03:00:56.801915 9025 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-kms:/tmp/dist-test-task1CUkeB/build/asan/bin/postgresql.jar:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1CUkeB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 03:00:56.801980 9025 mini_ranger_kms.cc:240] Using host: 127.8.208.84
I20260929 03:00:56.818672 9025 mini_ranger_kms.cc:276] Ranger KMS PID: 21281
I20260929 03:00:56.818809 9025 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.208.84:45417
Sep 29, 2026 3:00:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 3:00:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 3:00:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 3:00:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 3:00:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 3:00:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 03:00:59.117256 9025 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.455s user 0.090s sys 0.275s
I20260929 03:00:59.117499 9025 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 03:00:59.117666 9025 mini_ranger_kms.cc:401] 127.8.208.84:45417/kms/v1/keys
Sep 29 03:00:59 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790650859, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.208.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 03:00:59 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650859, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.208.84@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
Sep 29 03:01:00 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650859, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.208.84@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
Sep 29 03:01:02 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:01:02.421255 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:35801
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:35801
--ranger_config_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.208.84:45417/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:02.753280 21332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:02.753548 21332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:02.753690 21332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:02.761085 21332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:01:02.761220 21332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:02.761338 21332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:02.761433 21332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:01:02.761534 21332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:01:02.771646 21332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.208.84:45417/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:35801
--ranger_config_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:35801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:02.772895 21332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:02.774236 21332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:02.787923 21338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:02.787902 21340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:02.788520 21332 server_base.cc:1034] running on GCE node
W20260929 03:01:02.788641 21337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:02.789510 21332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:02.790850 21332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:02.792054 21332 hybrid_clock.cc:648] HybridClock initialized: now 1790650862792000 us; error 51 us; skew 500 ppm
I20260929 03:01:02.792375 21332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:02 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650862, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:02.795951 21332 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:01:02.797886 21332 webserver.cc:460] Webserver started at http://127.8.208.126:34179/ using document root <none> and password file <none>
I20260929 03:01:02.798763 21332 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:02.798911 21332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:02.799314 21332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:01:02 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650862, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:01:02.848449 21332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "46637da572074c46bdbd3adf78f0170e"
format_stamp: "Formatted at 2026-09-29 03:01:02 on dist-test-slave-m1rv"
server_key: "2508c235cb4d7906da955e4a4c9b251a"
server_key_iv: "ac49eb1a976d4ee06614fe66dba3c0e8"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:02.849444 21332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "46637da572074c46bdbd3adf78f0170e"
format_stamp: "Formatted at 2026-09-29 03:01:02 on dist-test-slave-m1rv"
server_key: "2508c235cb4d7906da955e4a4c9b251a"
server_key_iv: "ac49eb1a976d4ee06614fe66dba3c0e8"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:02.855123 21332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 03:01:02.871582 21347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:02.873332 21332 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 03:01:02.873505 21332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "46637da572074c46bdbd3adf78f0170e"
format_stamp: "Formatted at 2026-09-29 03:01:02 on dist-test-slave-m1rv"
server_key: "2508c235cb4d7906da955e4a4c9b251a"
server_key_iv: "ac49eb1a976d4ee06614fe66dba3c0e8"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:02.873742 21332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:02.906983 21332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:02.911106 21332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:02.911435 21332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:02.931407 21332 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:35801
I20260929 03:01:02.931407 21398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:35801 every 8 connection(s)
I20260929 03:01:02.932798 21332 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:01:02.933977 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 21332
I20260929 03:01:02.934180 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:01:02.935005 9025 external_mini_cluster.cc:1428] Setting key 0f22e81fe167532cf0bf746066b10f30
I20260929 03:01:02.938542 21399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:02.953421 21399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e: Bootstrap starting.
Sep 29 03:01:02 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:02.957245 21399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:02.958726 21399 log.cc:824] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:02.962019 21399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e: No bootstrap required, opened a new log
I20260929 03:01:02.965072 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:02.937663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56586 (local address 127.8.208.126:35801)
0929 03:01:02.941260 (+ 3597us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:02.941275 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:02.941324 (+ 49us) server_negotiation.cc:407] Connection header received
0929 03:01:02.943731 (+ 2407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:02.943761 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:02.944277 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:02.945332 (+ 1055us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:02.946345 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:02.948304 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:02.950107 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:02.951225 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:02.951419 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:02.956178 (+ 4759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:02.956188 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:02.956207 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:02.956254 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:02.960215 (+ 3961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:02.961045 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:02.961053 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:02.961062 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:02.961271 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:02.961700 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:02.961705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:02.961709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:02.962340 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:02.962619 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:02.963134 (+ 515us) server_negotiation.cc:299] Negotiation successful
0929 03:01:02.963716 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3353,"thread_start_us":3138,"threads_started":1}
I20260929 03:01:02.970626 21399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46637da572074c46bdbd3adf78f0170e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35801 } }
I20260929 03:01:02.970999 21399 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:02.971112 21399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46637da572074c46bdbd3adf78f0170e, State: Initialized, Role: FOLLOWER
I20260929 03:01:02.971650 21399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46637da572074c46bdbd3adf78f0170e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35801 } }
I20260929 03:01:02.971881 21399 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:01:02.972018 21399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:01:02.972199 21399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:02.974771 21399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46637da572074c46bdbd3adf78f0170e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35801 } }
I20260929 03:01:02.975234 21399 leader_election.cc:302] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46637da572074c46bdbd3adf78f0170e; no voters:
I20260929 03:01:02.975888 21399 leader_election.cc:288] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:01:02.976004 21404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:01:02.978590 21399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:01:02.983639 21404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 1 LEADER]: Becoming Leader. State: Replica: 46637da572074c46bdbd3adf78f0170e, State: Running, Role: LEADER
I20260929 03:01:02.984285 21404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46637da572074c46bdbd3adf78f0170e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35801 } }
I20260929 03:01:02.990751 21406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46637da572074c46bdbd3adf78f0170e. Latest consensus state: current_term: 1 leader_uuid: "46637da572074c46bdbd3adf78f0170e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46637da572074c46bdbd3adf78f0170e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35801 } } }
I20260929 03:01:02.991221 21406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:02.991853 21405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46637da572074c46bdbd3adf78f0170e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46637da572074c46bdbd3adf78f0170e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35801 } } }
I20260929 03:01:02.992154 21405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:02.994176 21418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:01:03.002810 21418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:01:03.007052 21399 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 03:01:03.022543 21418 catalog_manager.cc:1347] Generated new cluster ID: 78f38344d260457aa71d663efc9ab31d
I20260929 03:01:03.022845 21418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:01:03.058459 21418 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:01:03.060617 21418 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:01:03.075713 21418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e: Generated new TSK 0
I20260929 03:01:03.076974 21418 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 03:01:03.837244 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.208.84:45417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35801
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:01:04 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790650864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 03:01:04.262605 21445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:04.263167 21445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:04.263386 21445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:04.275174 21445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:04.275311 21445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:04.275549 21445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:01:04.291944 21445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.208.84:45417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:04.293787 21445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:04.295610 21445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:04.312247 21455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:04.312749 21456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:04.318461 21458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:04.319128 21445 server_base.cc:1034] running on GCE node
I20260929 03:01:04.320024 21445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:04.329417 21445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:04.330760 21445 hybrid_clock.cc:648] HybridClock initialized: now 1790650864330647 us; error 107 us; skew 500 ppm
I20260929 03:01:04.331241 21445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:04 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:04.335558 21445 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 03:01:04.338093 21445 webserver.cc:460] Webserver started at http://127.8.208.65:41219/ using document root <none> and password file <none>
I20260929 03:01:04.339039 21445 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:04.339178 21445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:04.339529 21445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:01:04 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:01:04.369486 21445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "01044b64599d4e1e9492dac347f2090a"
format_stamp: "Formatted at 2026-09-29 03:01:04 on dist-test-slave-m1rv"
server_key: "f8e0a84bb3103dab94f850269acef5b5"
server_key_iv: "80f8d6384e0819561bb66793e6ba974d"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:04.370568 21445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "01044b64599d4e1e9492dac347f2090a"
format_stamp: "Formatted at 2026-09-29 03:01:04 on dist-test-slave-m1rv"
server_key: "f8e0a84bb3103dab94f850269acef5b5"
server_key_iv: "80f8d6384e0819561bb66793e6ba974d"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:04.377470 21445 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260929 03:01:04.413858 21465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:04.418550 21445 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.010s sys 0.005s
I20260929 03:01:04.418762 21445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "01044b64599d4e1e9492dac347f2090a"
format_stamp: "Formatted at 2026-09-29 03:01:04 on dist-test-slave-m1rv"
server_key: "f8e0a84bb3103dab94f850269acef5b5"
server_key_iv: "80f8d6384e0819561bb66793e6ba974d"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:04.419063 21445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:04.452724 21445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:04.458290 21445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:04.458630 21445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:04.459847 21445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:04.461709 21445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:04.461807 21445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:04.461998 21445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:04.462072 21445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:04.552291 21445 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:37811
I20260929 03:01:04.552332 21577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:37811 every 8 connection(s)
I20260929 03:01:04.554133 21445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:01:04.558734 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 21445
I20260929 03:01:04.558976 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 03:01:04.559959 9025 external_mini_cluster.cc:1428] Setting key d2ca8261993a1781bed27a0cb0e4df9f
Sep 29 03:01:04 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
Sep 29 03:01:04 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:04.609270 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:04.556965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:53511 (local address 127.8.208.126:35801)
0929 03:01:04.576035 (+ 19070us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:04.576043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:04.585417 (+ 9374us) server_negotiation.cc:407] Connection header received
0929 03:01:04.588123 (+ 2706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:04.588130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:04.588227 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:04.588304 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:04.590518 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:04.591774 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:04.594549 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:04.595197 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:04.595330 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:04.601733 (+ 6403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:04.601739 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:04.601744 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:04.601784 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:04.605379 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:04.606375 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:04.606380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:04.606383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:04.606552 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:04.607097 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:04.607102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:04.607105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:04.607460 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:04.607587 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:04.608580 (+ 993us) server_negotiation.cc:299] Negotiation successful
0929 03:01:04.608814 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18883,"thread_start_us":19049,"threads_started":1}
I20260929 03:01:04.611799 21581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:04.557719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35801 (local address 127.8.208.65:53511)
0929 03:01:04.583210 (+ 25491us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:04.585268 (+ 2058us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:04.587781 (+ 2513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:04.588839 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:04.588852 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:04.589532 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:04.590324 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:04.590346 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:04.592048 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:04.592054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:04.594407 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:04.594421 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:04.595435 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:04.595442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:04.595713 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:04.596512 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:04.596558 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:04.601467 (+ 4909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:04.605558 (+ 4091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:04.605567 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:04.605585 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:04.606218 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:04.606702 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:04.606706 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:04.606709 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:04.606958 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:04.607686 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:04.607693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:04.608118 (+ 425us) client_negotiation.cc:769] Sending connection context
0929 03:01:04.609805 (+ 1687us) client_negotiation.cc:240] Negotiation successful
0929 03:01:04.610333 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25131,"thread_start_us":26468,"threads_started":1}
I20260929 03:01:04.617640 21578 heartbeater.cc:347] Connected to a master server at 127.8.208.126:35801
I20260929 03:01:04.617960 21578 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:04.618752 21578 heartbeater.cc:511] Master 127.8.208.126:35801 requested a full tablet report, sending...
I20260929 03:01:04.621232 21364 ts_manager.cc:194] Registered new tserver with Master: 01044b64599d4e1e9492dac347f2090a (127.8.208.65:37811)
I20260929 03:01:04.623236 21364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:53511
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
W20260929 03:01:04.680210 9025 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.208.66: real 0.117s user 0.000s sys 0.017s
I20260929 03:01:04.681128 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.208.84:45417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35801
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:01:04 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
W20260929 03:01:05.151484 21584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:05.151858 21584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:05.152014 21584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:05.159607 21584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:05.159744 21584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:05.159937 21584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:01:05.171948 21584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.208.84:45417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:05.173563 21584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:05.174834 21584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:05.197350 21591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:05.197371 21595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:05.201048 21592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:05.202754 21584 server_base.cc:1034] running on GCE node
I20260929 03:01:05.203768 21584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:05.205473 21584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:05.206811 21584 hybrid_clock.cc:648] HybridClock initialized: now 1790650865206700 us; error 99 us; skew 500 ppm
I20260929 03:01:05.207274 21584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:05 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650865, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:05.211865 21584 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 03:01:05.213773 21584 webserver.cc:460] Webserver started at http://127.8.208.66:36673/ using document root <none> and password file <none>
I20260929 03:01:05.214691 21584 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:05.214823 21584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:05.215229 21584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:01:05 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650865, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:01:05.232467 21584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "145189612cb14344957d9cc15df55a5f"
format_stamp: "Formatted at 2026-09-29 03:01:05 on dist-test-slave-m1rv"
server_key: "ed436c65b86a0e15ebfe97e896069555"
server_key_iv: "e9260a0671d85dd1340bca8438650c5c"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:05.233412 21584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "145189612cb14344957d9cc15df55a5f"
format_stamp: "Formatted at 2026-09-29 03:01:05 on dist-test-slave-m1rv"
server_key: "ed436c65b86a0e15ebfe97e896069555"
server_key_iv: "e9260a0671d85dd1340bca8438650c5c"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:05.243996 21584 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.004s
I20260929 03:01:05.261900 21608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:05.263546 21584 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260929 03:01:05.263732 21584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "145189612cb14344957d9cc15df55a5f"
format_stamp: "Formatted at 2026-09-29 03:01:05 on dist-test-slave-m1rv"
server_key: "ed436c65b86a0e15ebfe97e896069555"
server_key_iv: "e9260a0671d85dd1340bca8438650c5c"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:05.263973 21584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:05.296600 21584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:05.300971 21584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:05.301337 21584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:05.302495 21584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:05.303988 21584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:05.304100 21584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:05.304256 21584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:05.304332 21584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:05.347883 21584 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:41843
I20260929 03:01:05.347901 21720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:41843 every 8 connection(s)
I20260929 03:01:05.349323 21584 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:01:05.354341 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 21584
I20260929 03:01:05.354552 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 03:01:05.355227 9025 external_mini_cluster.cc:1428] Setting key c769464f9240243fc1d4bdc2bc2cbf7f
Sep 29 03:01:05 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650865, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:05.393802 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:05.352543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:46361 (local address 127.8.208.126:35801)
0929 03:01:05.368774 (+ 16231us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:05.368779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:05.368797 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:01:05.368849 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:05.368853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:05.368913 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:05.369077 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:05.371785 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:05.372965 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:05.375691 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:05.376458 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:05.376665 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:05.381614 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:05.381618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:05.381621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:05.381654 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:05.389066 (+ 7412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:05.390130 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:05.390139 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:05.390145 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:05.390302 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:05.390844 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:05.390852 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:05.390857 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:05.391295 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:05.391458 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:05.393073 (+ 1615us) server_negotiation.cc:299] Negotiation successful
0929 03:01:05.393327 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16116,"thread_start_us":16666,"threads_started":1}
I20260929 03:01:05.394659 21724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:05.353467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35801 (local address 127.8.208.66:46361)
0929 03:01:05.361938 (+ 8471us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:05.362015 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:05.364991 (+ 2976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:05.369394 (+ 4403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:05.369406 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:05.370709 (+ 1303us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:05.371572 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:05.371600 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:05.373164 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:05.373190 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:05.375474 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:05.375498 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:05.376968 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:05.376977 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:05.377400 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:05.378231 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:05.378326 (+ 95us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:05.381361 (+ 3035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:05.389275 (+ 7914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:05.389287 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:05.389308 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:05.389960 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:05.390435 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:05.390443 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:05.390450 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:05.390706 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:05.391535 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:05.391546 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:05.392012 (+ 466us) client_negotiation.cc:769] Sending connection context
0929 03:01:05.392361 (+ 349us) client_negotiation.cc:240] Negotiation successful
0929 03:01:05.392967 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8019,"thread_start_us":7725,"threads_started":1}
I20260929 03:01:05.396713 21721 heartbeater.cc:347] Connected to a master server at 127.8.208.126:35801
I20260929 03:01:05.397070 21721 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:05.397989 21721 heartbeater.cc:511] Master 127.8.208.126:35801 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
I20260929 03:01:05.399904 21364 ts_manager.cc:194] Registered new tserver with Master: 145189612cb14344957d9cc15df55a5f (127.8.208.66:41843)
I20260929 03:01:05.400856 21364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:46361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 03:01:05.430647 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.208.84:45417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35801
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:01:05.627027 21578 heartbeater.cc:503] Master 127.8.208.126:35801 was elected leader, sending a full tablet report...
W20260929 03:01:05.765398 21727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:05.765699 21727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:05.765863 21727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:05.773310 21727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:05.773445 21727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:05.773622 21727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:01:05.783615 21727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.208.84:45417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:05.784854 21727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:05.786168 21727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:05.798959 21735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:05.799005 21732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:05.799227 21733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:05.799726 21727 server_base.cc:1034] running on GCE node
I20260929 03:01:05.800279 21727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:05.801308 21727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:05.802524 21727 hybrid_clock.cc:648] HybridClock initialized: now 1790650865802481 us; error 53 us; skew 500 ppm
I20260929 03:01:05.802839 21727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:05 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650865, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:05.805872 21727 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 03:01:05.807188 21727 webserver.cc:460] Webserver started at http://127.8.208.67:41929/ using document root <none> and password file <none>
I20260929 03:01:05.807863 21727 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:05.808009 21727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:05.808310 21727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:01:05 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650865, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:01:05.824908 21727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "bd77967ad2484e2db1fe60745e667c84"
format_stamp: "Formatted at 2026-09-29 03:01:05 on dist-test-slave-m1rv"
server_key: "5c7d03b8e9387f638922454bc48f965e"
server_key_iv: "75bfa7ef792fd2023c034a53441a6249"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:05.825918 21727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "bd77967ad2484e2db1fe60745e667c84"
format_stamp: "Formatted at 2026-09-29 03:01:05 on dist-test-slave-m1rv"
server_key: "5c7d03b8e9387f638922454bc48f965e"
server_key_iv: "75bfa7ef792fd2023c034a53441a6249"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:05.832698 21727 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260929 03:01:05.849135 21742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:05.850903 21727 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:01:05.851082 21727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "bd77967ad2484e2db1fe60745e667c84"
format_stamp: "Formatted at 2026-09-29 03:01:05 on dist-test-slave-m1rv"
server_key: "5c7d03b8e9387f638922454bc48f965e"
server_key_iv: "75bfa7ef792fd2023c034a53441a6249"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:05.851336 21727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:05.876566 21727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:05.881824 21727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:05.882136 21727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:05.883114 21727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:05.884588 21727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:05.884660 21727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:05.884841 21727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:05.884891 21727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:05.917625 21727 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:42343
I20260929 03:01:05.917661 21854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:42343 every 8 connection(s)
I20260929 03:01:05.918915 21727 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:01:05.924837 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 21727
I20260929 03:01:05.925024 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 03:01:05.925557 9025 external_mini_cluster.cc:1428] Setting key 76572992c3125549a3086f61eea5bc74
Sep 29 03:01:05 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650865, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:05.938186 21857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:05.921521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:49765 (local address 127.8.208.126:35801)
0929 03:01:05.921948 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:05.921953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:05.922844 (+ 891us) server_negotiation.cc:407] Connection header received
0929 03:01:05.924714 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:05.924721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:05.924803 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:05.924923 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:05.926403 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:05.927194 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:05.928849 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:05.929326 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:05.929464 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:05.932559 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:05.932564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:05.932567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:05.932595 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:05.935260 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:05.935933 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:05.935940 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:05.935944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:05.936081 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:05.936519 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:05.936523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:05.936527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:05.936785 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:05.936942 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:05.937637 (+ 695us) server_negotiation.cc:299] Negotiation successful
0929 03:01:05.937848 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":214,"threads_started":1}
I20260929 03:01:05.939164 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:05.922039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35801 (local address 127.8.208.67:49765)
0929 03:01:05.922685 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:05.922727 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:05.924415 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:05.925215 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:05.925226 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:05.925663 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:05.926235 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:05.926251 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:05.927354 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:05.927359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:05.928704 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:05.928715 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:05.929540 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:05.929548 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:05.929779 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:05.930384 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:05.930404 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:05.932332 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:05.935408 (+ 3076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:05.935416 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:05.935430 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:05.935819 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:05.936184 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:05.936190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:05.936193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:05.936382 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:05.936920 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:05.936927 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:05.937283 (+ 356us) client_negotiation.cc:769] Sending connection context
0929 03:01:05.937533 (+ 250us) client_negotiation.cc:240] Negotiation successful
0929 03:01:05.938013 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":261,"threads_started":1}
I20260929 03:01:05.940716 21855 heartbeater.cc:347] Connected to a master server at 127.8.208.126:35801
I20260929 03:01:05.941030 21855 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:05.941700 21855 heartbeater.cc:511] Master 127.8.208.126:35801 requested a full tablet report, sending...
I20260929 03:01:05.943027 21363 ts_manager.cc:194] Registered new tserver with Master: bd77967ad2484e2db1fe60745e667c84 (127.8.208.67:42343)
I20260929 03:01:05.943884 21363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:49765
I20260929 03:01:05.952471 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:01:05.954218 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 21445
Sep 29 03:01:05 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 03:01:05.970803 21860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:05.955155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.8.208.65:37811)
0929 03:01:05.955755 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:05.955762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:05.955785 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:01:05.955838 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:05.955844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:05.956066 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:05.956177 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:05.957689 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:05.958750 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:05.960205 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:05.960711 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:05.960805 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:05.964919 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:05.964931 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:05.964945 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:05.964984 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:05.968150 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:05.968825 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:05.968832 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:05.968834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:05.969022 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:05.969397 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:05.969400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:05.969402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:05.969718 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:05.969938 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:05.970240 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 03:01:05.970436 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":305,"threads_started":1}
I20260929 03:01:05.972020 21573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:06.058249 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 21445
I20260929 03:01:06.074041 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 21584
Sep 29 03:01:06 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 03:01:06.090292 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:06.074969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47978 (local address 127.8.208.66:41843)
0929 03:01:06.075543 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:06.075550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:06.075576 (+ 26us) server_negotiation.cc:407] Connection header received
0929 03:01:06.075638 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:06.075645 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:06.075857 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:06.075988 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:06.077348 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:06.078445 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:06.079803 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:06.080334 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:06.080437 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:06.084522 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:06.084536 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:06.084550 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:06.084590 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:06.087743 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:06.088351 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:06.088358 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:06.088360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:06.088485 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:06.088794 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:06.088797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:06.088799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:06.089127 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:06.089518 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:06.089763 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 03:01:06.089960 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":312,"threads_started":1}
I20260929 03:01:06.091558 21716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:06.177034 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 21584
I20260929 03:01:06.193262 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 21727
Sep 29 03:01:06 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 03:01:06.209968 21864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:06.194361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40898 (local address 127.8.208.67:42343)
0929 03:01:06.194972 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:06.194980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:06.195009 (+ 29us) server_negotiation.cc:407] Connection header received
0929 03:01:06.195066 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:06.195072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:06.195284 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:06.195410 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:06.196750 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:06.197839 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:06.199430 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:06.200240 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:06.200338 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:06.204643 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:06.204653 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:06.204669 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:06.204701 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:06.207576 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:06.208206 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:06.208213 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:06.208215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:06.208331 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:06.208646 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:06.208648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:06.208650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:06.208957 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:06.209260 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:06.209511 (+ 251us) server_negotiation.cc:299] Negotiation successful
0929 03:01:06.209695 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":331,"threads_started":1}
I20260929 03:01:06.211236 21850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:06.294616 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 21727
I20260929 03:01:06.310138 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 21332
I20260929 03:01:06.311362 21394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:06.419569 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 21332
I20260929 03:01:06.441768 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:35801
--webserver_interface=127.8.208.126
--webserver_port=34179
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:35801
--ranger_config_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.208.84:45417/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.208.84:45417/kms with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:06.764227 21867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:06.764492 21867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:06.764684 21867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:06.772171 21867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:01:06.772301 21867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:06.772394 21867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:06.772490 21867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:01:06.772583 21867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:01:06.782642 21867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.208.84:45417/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:35801
--ranger_config_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:35801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=34179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:06.783845 21867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:06.785140 21867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:06.796854 21873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:06.796900 21872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:06.797405 21875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:06.797690 21867 server_base.cc:1034] running on GCE node
I20260929 03:01:06.798365 21867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:06.799628 21867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:06.800824 21867 hybrid_clock.cc:648] HybridClock initialized: now 1790650866800788 us; error 49 us; skew 500 ppm
I20260929 03:01:06.801107 21867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:06 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650866, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:06.804157 21867 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:01:06.805560 21867 webserver.cc:460] Webserver started at http://127.8.208.126:34179/ using document root <none> and password file <none>
I20260929 03:01:06.806123 21867 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:06.806221 21867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 03:01:06 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650866, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:01:06.864144 21867 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260929 03:01:06.868055 21883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:06.870118 21867 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 03:01:06.870317 21867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "46637da572074c46bdbd3adf78f0170e"
format_stamp: "Formatted at 2026-09-29 03:01:02 on dist-test-slave-m1rv"
server_key: "2508c235cb4d7906da955e4a4c9b251a"
server_key_iv: "ac49eb1a976d4ee06614fe66dba3c0e8"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:06.871897 21867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:06.889752 21867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:06.894105 21867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:06.894446 21867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:06.912482 21867 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:35801
I20260929 03:01:06.912503 21934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:35801 every 8 connection(s)
I20260929 03:01:06.914144 21867 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:01:06.920722 21935 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:01:06.923899 21935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e: Bootstrap starting.
I20260929 03:01:06.924034 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 21867
I20260929 03:01:06.924901 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.208.84:45417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:37811
--local_ip_for_outbound_sockets=127.8.208.65
--tserver_master_addrs=127.8.208.126:35801
--webserver_port=41219
--webserver_interface=127.8.208.65
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.208.84:45417/kms with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:01:06.942364 21935 log.cc:824] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:06.950994 21935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:01:06.951511 21935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e: Bootstrap complete.
I20260929 03:01:06.960534 21935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46637da572074c46bdbd3adf78f0170e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35801 } }
I20260929 03:01:06.961449 21935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46637da572074c46bdbd3adf78f0170e, State: Initialized, Role: FOLLOWER
I20260929 03:01:06.962105 21935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46637da572074c46bdbd3adf78f0170e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35801 } }
I20260929 03:01:06.962342 21935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:01:06.962459 21935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:01:06.962651 21935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 1 FOLLOWER]: Advancing to term 2
I20260929 03:01:06.966403 21935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46637da572074c46bdbd3adf78f0170e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35801 } }
I20260929 03:01:06.966878 21935 leader_election.cc:302] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46637da572074c46bdbd3adf78f0170e; no voters:
I20260929 03:01:06.967526 21935 leader_election.cc:288] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 03:01:06.967650 21939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 2 FOLLOWER]: Leader election won for term 2
I20260929 03:01:06.968745 21939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [term 2 LEADER]: Becoming Leader. State: Replica: 46637da572074c46bdbd3adf78f0170e, State: Running, Role: LEADER
I20260929 03:01:06.969374 21939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46637da572074c46bdbd3adf78f0170e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35801 } }
I20260929 03:01:06.969734 21935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:01:06.971890 21941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46637da572074c46bdbd3adf78f0170e. Latest consensus state: current_term: 2 leader_uuid: "46637da572074c46bdbd3adf78f0170e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46637da572074c46bdbd3adf78f0170e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35801 } } }
I20260929 03:01:06.972302 21941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:06.972932 21940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "46637da572074c46bdbd3adf78f0170e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46637da572074c46bdbd3adf78f0170e" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35801 } } }
I20260929 03:01:06.973287 21940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:06.978086 21945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:01:06.985215 21945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:01:06.985982 21935 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 03:01:06.986706 21945 catalog_manager.cc:1259] Loaded cluster ID: 78f38344d260457aa71d663efc9ab31d
I20260929 03:01:06.986861 21945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:01:06.990794 21945 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:01:06.992182 21945 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e: Loaded TSK: 0
I20260929 03:01:06.993710 21945 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 03:01:07.342581 21937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:07.343034 21937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:07.343245 21937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:07.352953 21937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:07.353114 21937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:07.353325 21937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:01:07.365319 21937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.208.84:45417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:37811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=41219
--tserver_master_addrs=127.8.208.126:35801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:07.367018 21937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:07.368763 21937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:07.402907 21972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:07.403887 21973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:07.424326 21937 server_base.cc:1034] running on GCE node
W20260929 03:01:07.424728 21975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:07.425601 21937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:07.427038 21937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:07.428308 21937 hybrid_clock.cc:648] HybridClock initialized: now 1790650867428239 us; error 65 us; skew 500 ppm
I20260929 03:01:07.428720 21937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:07 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:07.432612 21937 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 03:01:07.439599 21937 webserver.cc:460] Webserver started at http://127.8.208.65:41219/ using document root <none> and password file <none>
I20260929 03:01:07.440482 21937 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:07.440650 21937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 03:01:07 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:01:07.480314 21937 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 03:01:07.489979 21983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:07.492069 21937 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.007s
I20260929 03:01:07.492336 21937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "01044b64599d4e1e9492dac347f2090a"
format_stamp: "Formatted at 2026-09-29 03:01:04 on dist-test-slave-m1rv"
server_key: "f8e0a84bb3103dab94f850269acef5b5"
server_key_iv: "80f8d6384e0819561bb66793e6ba974d"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:07.493976 21937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:07.531401 21937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:07.536495 21937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:07.536873 21937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:07.538051 21937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:07.539638 21937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:07.539762 21937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:07.539901 21937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:07.540007 21937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:07.625574 21937 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:37811
I20260929 03:01:07.625599 22096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:37811 every 8 connection(s)
I20260929 03:01:07.627467 21937 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:01:07.636518 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 21937
I20260929 03:01:07.637414 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.208.84:45417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:41843
--local_ip_for_outbound_sockets=127.8.208.66
--tserver_master_addrs=127.8.208.126:35801
--webserver_port=36673
--webserver_interface=127.8.208.66
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.208.84:45417/kms with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:01:07 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:07.661993 22100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:07.630822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35801 (local address 127.8.208.65:41899)
0929 03:01:07.634689 (+ 3867us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:07.634753 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:07.637299 (+ 2546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:07.639682 (+ 2383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:07.639693 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:07.640330 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:07.641191 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:07.641221 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:07.643698 (+ 2477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:07.643703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:07.645846 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:07.645872 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:07.647616 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:07.647623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:07.647898 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:07.648720 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:07.648749 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:07.651562 (+ 2813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:07.656571 (+ 5009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:07.656581 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:07.656598 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:07.657246 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:07.657802 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:07.657809 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:07.657815 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:07.658384 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:07.659481 (+ 1097us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:07.659489 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:07.659867 (+ 378us) client_negotiation.cc:769] Sending connection context
0929 03:01:07.660158 (+ 291us) client_negotiation.cc:240] Negotiation successful
0929 03:01:07.660678 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3538,"thread_start_us":3350,"threads_started":1}
I20260929 03:01:07.662305 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:07.630718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:41899 (local address 127.8.208.126:35801)
0929 03:01:07.638305 (+ 7587us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:07.638319 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:07.638358 (+ 39us) server_negotiation.cc:407] Connection header received
0929 03:01:07.638772 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:07.638786 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:07.639167 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:07.639543 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:07.641484 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:07.643496 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:07.646054 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:07.647106 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:07.647289 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:07.651888 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:07.651899 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:07.651921 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:07.651979 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:07.656378 (+ 4399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:07.657425 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:07.657437 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:07.657448 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:07.657655 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:07.658535 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:07.658542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:07.658549 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:07.659301 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:07.659550 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:07.660458 (+ 908us) server_negotiation.cc:299] Negotiation successful
0929 03:01:07.660898 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7301,"thread_start_us":13678,"threads_started":1}
I20260929 03:01:07.665948 22097 heartbeater.cc:347] Connected to a master server at 127.8.208.126:35801
I20260929 03:01:07.666255 22097 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:07.667054 22097 heartbeater.cc:511] Master 127.8.208.126:35801 requested a full tablet report, sending...
W20260929 03:01:07.670339 22097 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:35801 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 03:01:07.670576 22097 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:07.670849 22097 heartbeater.cc:511] Master 127.8.208.126:35801 requested a full tablet report, sending...
I20260929 03:01:07.671991 22097 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:07.672221 22097 heartbeater.cc:511] Master 127.8.208.126:35801 requested a full tablet report, sending...
W20260929 03:01:07.673241 22097 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 03:01:08.039188 22101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:08.039507 22101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:08.039675 22101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:08.047899 22101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:08.048040 22101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:08.048274 22101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:01:08.058674 22101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.208.84:45417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:41843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=36673
--tserver_master_addrs=127.8.208.126:35801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:08.059958 22101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:08.061259 22101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:08.078861 22114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:08.079414 22116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:08.083714 22118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:08.083866 22101 server_base.cc:1034] running on GCE node
I20260929 03:01:08.084492 22101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:08.085626 22101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:08.086908 22101 hybrid_clock.cc:648] HybridClock initialized: now 1790650868086816 us; error 78 us; skew 500 ppm
I20260929 03:01:08.087242 22101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:08 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650868, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:08.095091 22101 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 03:01:08.097052 22101 webserver.cc:460] Webserver started at http://127.8.208.66:36673/ using document root <none> and password file <none>
I20260929 03:01:08.097805 22101 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:08.097961 22101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 03:01:08 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650868, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:01:08.118973 22101 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 03:01:08.122561 22126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:08.124480 22101 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260929 03:01:08.124758 22101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "145189612cb14344957d9cc15df55a5f"
format_stamp: "Formatted at 2026-09-29 03:01:05 on dist-test-slave-m1rv"
server_key: "ed436c65b86a0e15ebfe97e896069555"
server_key_iv: "e9260a0671d85dd1340bca8438650c5c"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:08.126163 22101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:08.213116 22101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:08.218449 22101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:08.218775 22101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:08.220070 22101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:08.221859 22101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:08.221973 22101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:08.222079 22101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:08.222154 22101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 03:01:08 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790650868, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:08.310011 22101 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:41843
I20260929 03:01:08.310168 22240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:41843 every 8 connection(s)
I20260929 03:01:08.311883 22101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:01:08.316260 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 22101
I20260929 03:01:08.317090 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.208.84:45417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:42343
--local_ip_for_outbound_sockets=127.8.208.67
--tserver_master_addrs=127.8.208.126:35801
--webserver_port=41929
--webserver_interface=127.8.208.67
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.208.84:45417/kms with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:01:08 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650868, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:08.352476 22244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:08.319683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:38039 (local address 127.8.208.126:35801)
0929 03:01:08.320168 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:08.320175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:08.329992 (+ 9817us) server_negotiation.cc:407] Connection header received
0929 03:01:08.333729 (+ 3737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:08.333736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:08.333829 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:08.333914 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:08.336150 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:08.337393 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:08.339760 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:08.340393 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:08.340491 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:08.345228 (+ 4737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:08.345235 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:08.345240 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:08.345281 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:08.348851 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:08.349756 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:08.349762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:08.349766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:08.349907 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:08.350443 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:08.350448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:08.350451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:08.350798 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:08.350931 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:08.351877 (+ 946us) server_negotiation.cc:299] Negotiation successful
0929 03:01:08.352095 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":360,"threads_started":1}
I20260929 03:01:08.354930 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:08.315324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35801 (local address 127.8.208.66:38039)
0929 03:01:08.329786 (+ 14462us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:08.329841 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:08.333351 (+ 3510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:08.334479 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:08.334492 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:08.335141 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:08.335927 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:08.335951 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:08.337591 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:08.337597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:08.339582 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:08.339606 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:08.340827 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:08.340834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:08.341117 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:08.342203 (+ 1086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:08.342238 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:08.344959 (+ 2721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:08.349027 (+ 4068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:08.349036 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:08.349054 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:08.349621 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:08.350059 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:08.350064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:08.350068 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:08.350309 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:08.351021 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:08.351029 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:08.351425 (+ 396us) client_negotiation.cc:769] Sending connection context
0929 03:01:08.353041 (+ 1616us) client_negotiation.cc:240] Negotiation successful
0929 03:01:08.353600 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14095,"thread_start_us":14585,"threads_started":1}
I20260929 03:01:08.356879 22241 heartbeater.cc:347] Connected to a master server at 127.8.208.126:35801
I20260929 03:01:08.357203 22241 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:08.358012 22241 heartbeater.cc:511] Master 127.8.208.126:35801 requested a full tablet report, sending...
I20260929 03:01:08.359921 21900 ts_manager.cc:194] Registered new tserver with Master: 145189612cb14344957d9cc15df55a5f (127.8.208.66:41843)
I20260929 03:01:08.362190 21900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:38039
I20260929 03:01:08.673352 22097 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:08.673866 22097 heartbeater.cc:511] Master 127.8.208.126:35801 requested a full tablet report, sending...
I20260929 03:01:08.675341 21900 ts_manager.cc:194] Registered new tserver with Master: 01044b64599d4e1e9492dac347f2090a (127.8.208.65:37811)
I20260929 03:01:08.676602 21900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:41899
Sep 29 03:01:08 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650868, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
W20260929 03:01:08.763032 22245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:08.763370 22245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:08.763538 22245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:08.774480 22245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:08.774719 22245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:08.774955 22245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:01:08.790499 22245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.208.84:45417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:42343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=41929
--tserver_master_addrs=127.8.208.126:35801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:08.792235 22245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:08.793936 22245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:08.809338 22251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:08.811086 22252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:08.812209 22254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:08.812974 22245 server_base.cc:1034] running on GCE node
I20260929 03:01:08.813659 22245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:08.814772 22245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:08.816015 22245 hybrid_clock.cc:648] HybridClock initialized: now 1790650868815951 us; error 44 us; skew 500 ppm
I20260929 03:01:08.816376 22245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:08 dist-test-slave-m1rv krb5kdc[20888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650868, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:08.820518 22245 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 03:01:08.822358 22245 webserver.cc:460] Webserver started at http://127.8.208.67:41929/ using document root <none> and password file <none>
I20260929 03:01:08.823266 22245 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:08.823432 22245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 03:01:08 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650868, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:01:08.846254 22245 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260929 03:01:08.849864 22262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:08.851617 22245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:01:08.851799 22245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "bd77967ad2484e2db1fe60745e667c84"
format_stamp: "Formatted at 2026-09-29 03:01:05 on dist-test-slave-m1rv"
server_key: "5c7d03b8e9387f638922454bc48f965e"
server_key_iv: "75bfa7ef792fd2023c034a53441a6249"
server_key_version: "kuduclusterkey@0"
I20260929 03:01:08.853400 22245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:08.873975 22245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:08.878371 22245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:08.878710 22245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:08.879931 22245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:08.881476 22245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:08.881592 22245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:08.881728 22245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:08.881814 22245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:08.951169 22245 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:42343
I20260929 03:01:08.951223 22374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:42343 every 8 connection(s)
I20260929 03:01:08.952694 22245 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:01:08.958390 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 22245
Sep 29 03:01:08 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650868, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:08.994445 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:08.959761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56526 (local address 127.8.208.126:35801)
0929 03:01:08.974274 (+ 14513us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:08.974280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:08.974301 (+ 21us) server_negotiation.cc:407] Connection header received
0929 03:01:08.974369 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:08.974374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:08.974474 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:08.974605 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:08.977287 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:08.978504 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:08.980507 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:08.981131 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:08.981279 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:08.985467 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:08.985474 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:08.985479 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:08.985520 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:08.990245 (+ 4725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:08.991166 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:08.991172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:08.991175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:08.991337 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:08.991912 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:08.991917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:08.991920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:08.992291 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:08.992543 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:08.993802 (+ 1259us) server_negotiation.cc:299] Negotiation successful
0929 03:01:08.994029 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14378,"thread_start_us":10950,"threads_started":1}
I20260929 03:01:08.994737 22380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:08.956984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:40041 (local address 127.8.208.126:35801)
0929 03:01:08.970968 (+ 13984us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:08.970974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:08.970993 (+ 19us) server_negotiation.cc:407] Connection header received
0929 03:01:08.975779 (+ 4786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:08.975786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:08.975869 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:08.975967 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:08.978241 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:08.979502 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:08.982002 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:08.982688 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:08.982787 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:08.987284 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:08.987291 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:08.987296 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:08.987357 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:08.991024 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:08.991969 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:08.991974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:08.991977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:08.992115 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:08.992649 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:08.992654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:08.992657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:08.992973 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:08.993098 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:08.994113 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0929 03:01:08.994331 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13843,"thread_start_us":16985,"threads_started":1}
I20260929 03:01:08.997376 22378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:08.957643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35801 (local address 127.8.208.67:40041)
0929 03:01:08.970093 (+ 12450us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:08.970154 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:08.975302 (+ 5148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:08.976471 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:08.976484 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:08.977119 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:08.978035 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:08.978062 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:08.979692 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:08.979699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:08.981835 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:08.981849 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:08.983147 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:08.983154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:08.983461 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:08.984239 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:08.984269 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:08.987005 (+ 2736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:08.991185 (+ 4180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:08.991195 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:08.991213 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:08.991830 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:08.992251 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:08.992255 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:08.992259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:08.992513 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:08.993221 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:08.993229 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:08.993659 (+ 430us) client_negotiation.cc:769] Sending connection context
0929 03:01:08.994805 (+ 1146us) client_negotiation.cc:240] Negotiation successful
0929 03:01:08.995880 (+ 1075us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12111,"thread_start_us":11878,"threads_started":1}
I20260929 03:01:08.999187 22375 heartbeater.cc:347] Connected to a master server at 127.8.208.126:35801
I20260929 03:01:08.999534 22375 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:09.000402 22375 heartbeater.cc:511] Master 127.8.208.126:35801 requested a full tablet report, sending...
I20260929 03:01:09.002120 21900 ts_manager.cc:194] Registered new tserver with Master: bd77967ad2484e2db1fe60745e667c84 (127.8.208.67:42343)
I20260929 03:01:09.003252 21900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:40041
I20260929 03:01:09.011996 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:01:09.050802 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.031376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56530 (local address 127.8.208.126:35801)
0929 03:01:09.031569 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:09.031575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:09.033784 (+ 2209us) server_negotiation.cc:407] Connection header received
0929 03:01:09.034000 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:09.034006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:09.034095 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:09.034227 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:09.036288 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.037485 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.039521 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.040162 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.040264 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.043580 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:09.043587 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:09.043592 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:09.043637 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:09.046848 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:09.047989 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:09.047995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:09.047998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:09.048162 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:09.049224 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:09.049234 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:09.049237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:09.049630 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:09.049894 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:09.050230 (+ 336us) server_negotiation.cc:299] Negotiation successful
0929 03:01:09.050455 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Sep 29 03:01:09 dist-test-slave-m1rv krb5kdc[20888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650868, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for HTTP/127.8.208.84@KRBTEST.COM
I20260929 03:01:09.061771 21900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:01:09.071616 21900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:01:09.128978 22391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.100159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:37811 (local address 127.0.0.1:38558)
0929 03:01:09.117698 (+ 17539us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:09.117762 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:09.118003 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:09.119384 (+ 1381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:09.119392 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:09.119438 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:09.119822 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:09.119836 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.122144 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.122153 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:09.125117 (+ 2964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:09.125131 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.127660 (+ 2529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.127670 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:09.127969 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.128039 (+ 70us) client_negotiation.cc:769] Sending connection context
0929 03:01:09.128191 (+ 152us) client_negotiation.cc:240] Negotiation successful
0929 03:01:09.128458 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17237,"thread_start_us":17094,"threads_started":1}
I20260929 03:01:09.130103 22392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.100549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38558 (local address 127.8.208.65:37811)
0929 03:01:09.117024 (+ 16475us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:09.117034 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:09.118455 (+ 1421us) server_negotiation.cc:407] Connection header received
0929 03:01:09.118561 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:09.118569 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:09.118891 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:09.119087 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:09.120046 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.121943 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.125343 (+ 3400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.126977 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.127116 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.127422 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:09.129595 (+ 2173us) server_negotiation.cc:299] Negotiation successful
0929 03:01:09.129767 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16266,"thread_start_us":17605,"threads_started":1}
I20260929 03:01:09.134785 22031 tablet_service.cc:1467] Processing CreateTablet for tablet 23c15a8cfeff439f807f8f39bf0e8860 (DEFAULT_TABLE table=test-table [id=8c39b053254245cf99ce7bae29c09629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:01:09.136478 22031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23c15a8cfeff439f807f8f39bf0e8860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:09.145305 22391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.109650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:42343 (local address 127.0.0.1:46488)
0929 03:01:09.129283 (+ 19633us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:09.129303 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:09.129430 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:09.135328 (+ 5898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:09.135337 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:09.135378 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:09.135650 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:09.135665 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.139047 (+ 3382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.139052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:09.141578 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:09.141600 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.144474 (+ 2874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.144481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:09.144690 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.144726 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 03:01:09.144828 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 03:01:09.144992 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19503,"threads_started":1}
I20260929 03:01:09.146140 22394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.110512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46488 (local address 127.8.208.67:42343)
0929 03:01:09.134645 (+ 24133us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:09.134654 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:09.134686 (+ 32us) server_negotiation.cc:407] Connection header received
0929 03:01:09.134811 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:09.134820 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:09.135072 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:09.135285 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:09.135892 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.138144 (+ 2252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.141840 (+ 3696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.143856 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.144016 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.144310 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:09.145551 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0929 03:01:09.145768 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23957,"thread_start_us":23793,"threads_started":1}
I20260929 03:01:09.151296 22310 tablet_service.cc:1467] Processing CreateTablet for tablet 23c15a8cfeff439f807f8f39bf0e8860 (DEFAULT_TABLE table=test-table [id=8c39b053254245cf99ce7bae29c09629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:01:09.154152 22310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23c15a8cfeff439f807f8f39bf0e8860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:09.155800 22393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.147098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:41843 (local address 127.0.0.1:43942)
0929 03:01:09.147290 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:09.147310 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:09.147456 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:09.148384 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:09.148389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:09.148431 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:09.148780 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:09.148791 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.151038 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.151043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:09.152988 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:09.153003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.155022 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.155046 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:09.155216 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.155248 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 03:01:09.155342 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 03:01:09.155487 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 03:01:09.156502 22395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.146475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43942 (local address 127.8.208.66:41843)
0929 03:01:09.147007 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:09.147015 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:09.147588 (+ 573us) server_negotiation.cc:407] Connection header received
0929 03:01:09.147696 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:09.147703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:09.147946 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:09.148147 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:09.148989 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.150854 (+ 1865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.153210 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.154578 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.154699 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.154877 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:09.156016 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0929 03:01:09.156189 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":286,"threads_started":1}
I20260929 03:01:09.160907 22176 tablet_service.cc:1467] Processing CreateTablet for tablet 23c15a8cfeff439f807f8f39bf0e8860 (DEFAULT_TABLE table=test-table [id=8c39b053254245cf99ce7bae29c09629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:01:09.162372 22176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23c15a8cfeff439f807f8f39bf0e8860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:09.178565 22397 tablet_bootstrap.cc:492] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84: Bootstrap starting.
I20260929 03:01:09.184823 22396 tablet_bootstrap.cc:492] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a: Bootstrap starting.
I20260929 03:01:09.189610 22396 tablet_bootstrap.cc:654] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:09.190627 22397 tablet_bootstrap.cc:654] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:09.192487 22397 log.cc:824] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:09.193492 22398 tablet_bootstrap.cc:492] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f: Bootstrap starting.
I20260929 03:01:09.197300 22398 tablet_bootstrap.cc:654] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:09.199265 22398 log.cc:824] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:09.206614 22396 log.cc:824] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:09.208092 22397 tablet_bootstrap.cc:492] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84: No bootstrap required, opened a new log
I20260929 03:01:09.208485 22397 ts_tablet_manager.cc:1397] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84: Time spent bootstrapping tablet: real 0.030s user 0.016s sys 0.000s
I20260929 03:01:09.210927 22398 tablet_bootstrap.cc:492] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f: No bootstrap required, opened a new log
I20260929 03:01:09.211303 22398 ts_tablet_manager.cc:1397] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.010s
I20260929 03:01:09.219462 22397 raft_consensus.cc:348] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01044b64599d4e1e9492dac347f2090a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37811 } } peers { permanent_uuid: "bd77967ad2484e2db1fe60745e667c84" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "145189612cb14344957d9cc15df55a5f" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41843 } }
I20260929 03:01:09.220014 22397 raft_consensus.cc:374] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:09.220153 22397 raft_consensus.cc:729] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd77967ad2484e2db1fe60745e667c84, State: Initialized, Role: FOLLOWER
I20260929 03:01:09.220882 22397 consensus_queue.cc:260] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01044b64599d4e1e9492dac347f2090a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37811 } } peers { permanent_uuid: "bd77967ad2484e2db1fe60745e667c84" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "145189612cb14344957d9cc15df55a5f" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41843 } }
I20260929 03:01:09.221350 22396 tablet_bootstrap.cc:492] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a: No bootstrap required, opened a new log
I20260929 03:01:09.221719 22396 ts_tablet_manager.cc:1397] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a: Time spent bootstrapping tablet: real 0.037s user 0.010s sys 0.000s
I20260929 03:01:09.224942 22398 raft_consensus.cc:348] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01044b64599d4e1e9492dac347f2090a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37811 } } peers { permanent_uuid: "bd77967ad2484e2db1fe60745e667c84" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "145189612cb14344957d9cc15df55a5f" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41843 } }
I20260929 03:01:09.225643 22398 raft_consensus.cc:374] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:09.225962 22398 raft_consensus.cc:729] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145189612cb14344957d9cc15df55a5f, State: Initialized, Role: FOLLOWER
I20260929 03:01:09.226972 22398 consensus_queue.cc:260] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01044b64599d4e1e9492dac347f2090a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37811 } } peers { permanent_uuid: "bd77967ad2484e2db1fe60745e667c84" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "145189612cb14344957d9cc15df55a5f" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41843 } }
I20260929 03:01:09.230530 22375 heartbeater.cc:503] Master 127.8.208.126:35801 was elected leader, sending a full tablet report...
I20260929 03:01:09.230940 22397 ts_tablet_manager.cc:1428] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84: Time spent starting tablet: real 0.022s user 0.013s sys 0.009s
I20260929 03:01:09.234146 22398 ts_tablet_manager.cc:1428] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f: Time spent starting tablet: real 0.023s user 0.015s sys 0.001s
I20260929 03:01:09.234576 22241 heartbeater.cc:503] Master 127.8.208.126:35801 was elected leader, sending a full tablet report...
I20260929 03:01:09.241292 22396 raft_consensus.cc:348] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01044b64599d4e1e9492dac347f2090a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37811 } } peers { permanent_uuid: "bd77967ad2484e2db1fe60745e667c84" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "145189612cb14344957d9cc15df55a5f" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41843 } }
I20260929 03:01:09.241922 22396 raft_consensus.cc:374] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:09.242079 22396 raft_consensus.cc:729] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01044b64599d4e1e9492dac347f2090a, State: Initialized, Role: FOLLOWER
I20260929 03:01:09.242799 22396 consensus_queue.cc:260] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01044b64599d4e1e9492dac347f2090a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37811 } } peers { permanent_uuid: "bd77967ad2484e2db1fe60745e667c84" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "145189612cb14344957d9cc15df55a5f" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41843 } }
I20260929 03:01:09.251137 22396 ts_tablet_manager.cc:1428] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a: Time spent starting tablet: real 0.029s user 0.007s sys 0.012s
I20260929 03:01:09.251605 22097 heartbeater.cc:503] Master 127.8.208.126:35801 was elected leader, sending a full tablet report...
W20260929 03:01:09.319950 22242 tablet.cc:2369] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:01:09.389254 22098 tablet.cc:2369] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:01:09.457376 22376 tablet.cc:2369] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:01:09.475010 22402 raft_consensus.cc:482] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:01:09.475350 22402 raft_consensus.cc:504] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01044b64599d4e1e9492dac347f2090a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37811 } } peers { permanent_uuid: "bd77967ad2484e2db1fe60745e667c84" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "145189612cb14344957d9cc15df55a5f" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41843 } }
I20260929 03:01:09.477293 22402 leader_election.cc:288] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01044b64599d4e1e9492dac347f2090a (127.8.208.65:37811), 145189612cb14344957d9cc15df55a5f (127.8.208.66:41843)
I20260929 03:01:09.484578 22392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.478145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:41615 (local address 127.8.208.65:37811)
0929 03:01:09.478354 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:09.478359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:09.478378 (+ 19us) server_negotiation.cc:407] Connection header received
0929 03:01:09.478455 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:09.478460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:09.478559 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:09.478648 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:09.479115 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.480119 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.482475 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.483600 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.483750 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.483819 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:09.484187 (+ 368us) server_negotiation.cc:299] Negotiation successful
0929 03:01:09.484312 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 03:01:09.484575 22378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.477552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:37811 (local address 127.8.208.67:41615)
0929 03:01:09.478035 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:09.478057 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:09.478182 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:09.478698 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:09.478703 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:09.478731 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:09.478956 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:09.478967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.480242 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.480249 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:09.482309 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:09.482323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.483762 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.483770 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:09.483987 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.484025 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 03:01:09.484149 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 03:01:09.484279 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":32}
I20260929 03:01:09.484736 22395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.477915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:59127 (local address 127.8.208.66:41843)
0929 03:01:09.478130 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:09.478136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:09.478584 (+ 448us) server_negotiation.cc:407] Connection header received
0929 03:01:09.478786 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:09.478795 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:09.478883 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:09.478948 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:09.479721 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.480779 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.482920 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.483734 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.483873 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.483952 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:09.484342 (+ 390us) server_negotiation.cc:299] Negotiation successful
0929 03:01:09.484492 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260929 03:01:09.485423 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.477679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:41843 (local address 127.8.208.67:59127)
0929 03:01:09.478461 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:09.478479 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:09.478616 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:09.479139 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:09.479143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:09.479169 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:09.479553 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:09.479565 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.480955 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.480960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:09.482783 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:09.482793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.483985 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.483991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:09.484168 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.484198 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 03:01:09.484900 (+ 702us) client_negotiation.cc:240] Negotiation successful
0929 03:01:09.485045 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":5,"thread_start_us":1093,"threads_started":1}
I20260929 03:01:09.486115 22051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23c15a8cfeff439f807f8f39bf0e8860" candidate_uuid: "bd77967ad2484e2db1fe60745e667c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01044b64599d4e1e9492dac347f2090a" is_pre_election: true
I20260929 03:01:09.486498 22196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23c15a8cfeff439f807f8f39bf0e8860" candidate_uuid: "bd77967ad2484e2db1fe60745e667c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145189612cb14344957d9cc15df55a5f" is_pre_election: true
I20260929 03:01:09.486630 22051 raft_consensus.cc:2454] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd77967ad2484e2db1fe60745e667c84 in term 0.
I20260929 03:01:09.487017 22196 raft_consensus.cc:2454] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd77967ad2484e2db1fe60745e667c84 in term 0.
I20260929 03:01:09.487331 22264 leader_election.cc:302] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01044b64599d4e1e9492dac347f2090a, bd77967ad2484e2db1fe60745e667c84; no voters:
I20260929 03:01:09.487788 22402 raft_consensus.cc:2781] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:01:09.487932 22402 raft_consensus.cc:482] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:01:09.488062 22402 raft_consensus.cc:3037] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:09.490713 22402 raft_consensus.cc:504] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01044b64599d4e1e9492dac347f2090a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37811 } } peers { permanent_uuid: "bd77967ad2484e2db1fe60745e667c84" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "145189612cb14344957d9cc15df55a5f" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41843 } }
I20260929 03:01:09.491343 22402 leader_election.cc:288] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [CANDIDATE]: Term 1 election: Requested vote from peers 01044b64599d4e1e9492dac347f2090a (127.8.208.65:37811), 145189612cb14344957d9cc15df55a5f (127.8.208.66:41843)
I20260929 03:01:09.491729 22051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23c15a8cfeff439f807f8f39bf0e8860" candidate_uuid: "bd77967ad2484e2db1fe60745e667c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01044b64599d4e1e9492dac347f2090a"
I20260929 03:01:09.491838 22196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23c15a8cfeff439f807f8f39bf0e8860" candidate_uuid: "bd77967ad2484e2db1fe60745e667c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145189612cb14344957d9cc15df55a5f"
I20260929 03:01:09.491967 22051 raft_consensus.cc:3037] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:09.492039 22196 raft_consensus.cc:3037] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:09.495232 22051 raft_consensus.cc:2454] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd77967ad2484e2db1fe60745e667c84 in term 1.
I20260929 03:01:09.495232 22196 raft_consensus.cc:2454] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd77967ad2484e2db1fe60745e667c84 in term 1.
I20260929 03:01:09.495711 22264 leader_election.cc:302] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01044b64599d4e1e9492dac347f2090a, bd77967ad2484e2db1fe60745e667c84; no voters:
I20260929 03:01:09.496058 22402 raft_consensus.cc:2781] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:01:09.496577 22402 raft_consensus.cc:686] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 1 LEADER]: Becoming Leader. State: Replica: bd77967ad2484e2db1fe60745e667c84, State: Running, Role: LEADER
I20260929 03:01:09.497108 22402 consensus_queue.cc:237] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01044b64599d4e1e9492dac347f2090a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37811 } } peers { permanent_uuid: "bd77967ad2484e2db1fe60745e667c84" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "145189612cb14344957d9cc15df55a5f" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41843 } }
I20260929 03:01:09.503017 21899 catalog_manager.cc:5495] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd77967ad2484e2db1fe60745e667c84 (127.8.208.67). New cstate: current_term: 1 leader_uuid: "bd77967ad2484e2db1fe60745e667c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01044b64599d4e1e9492dac347f2090a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd77967ad2484e2db1fe60745e667c84" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145189612cb14344957d9cc15df55a5f" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41843 } health_report { overall_health: UNKNOWN } } }
I20260929 03:01:09.590111 22394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.584850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.8.208.67:42343)
0929 03:01:09.585032 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:09.585037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:09.585407 (+ 370us) server_negotiation.cc:407] Connection header received
0929 03:01:09.585573 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:09.585577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:09.585653 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:09.585798 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:01:09.586133 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.586823 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.588206 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.588613 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.588703 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.589141 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:01:09.589672 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:01:09.589776 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:09.589853 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 03:01:09.589913 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 03:01:09.598232 22051 raft_consensus.cc:1261] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a [term 1 FOLLOWER]: Refusing update from remote peer bd77967ad2484e2db1fe60745e667c84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:01:09.598227 22196 raft_consensus.cc:1261] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f [term 1 FOLLOWER]: Refusing update from remote peer bd77967ad2484e2db1fe60745e667c84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:01:09.599336 22413 consensus_queue.cc:1035] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145189612cb14344957d9cc15df55a5f" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:01:09.599702 22402 consensus_queue.cc:1035] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01044b64599d4e1e9492dac347f2090a" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 37811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:01:09.611845 22417 mvcc.cc:205] Tried to move back new op lower bound from 7334505961863839744 to 7334505961464758272. Current Snapshot: MvccSnapshot[applied={T|T < 7334505961863839744}]
I20260929 03:01:09.613318 22418 mvcc.cc:205] Tried to move back new op lower bound from 7334505961863839744 to 7334505961464758272. Current Snapshot: MvccSnapshot[applied={T|T < 7334505961863839744}]
I20260929 03:01:09.625320 21899 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:01:09.625614 21899 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:01:09.628854 21899 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46637da572074c46bdbd3adf78f0170e: Sending DeleteTablet for 3 replicas of tablet 23c15a8cfeff439f807f8f39bf0e8860
I20260929 03:01:09.629979 22031 tablet_service.cc:1514] Processing DeleteTablet for tablet 23c15a8cfeff439f807f8f39bf0e8860 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:01:09 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:38558
I20260929 03:01:09.629980 22310 tablet_service.cc:1514] Processing DeleteTablet for tablet 23c15a8cfeff439f807f8f39bf0e8860 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:01:09 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:46488
I20260929 03:01:09.633051 22176 tablet_service.cc:1514] Processing DeleteTablet for tablet 23c15a8cfeff439f807f8f39bf0e8860 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:01:09 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:43942
I20260929 03:01:09.639431 22429 tablet_replica.cc:330] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f: stopping tablet replica
I20260929 03:01:09.641371 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 21937
I20260929 03:01:09.645501 22429 raft_consensus.cc:2229] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:01:09.646875 22429 raft_consensus.cc:2258] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:01:09.647675 22428 tablet_replica.cc:330] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84: stopping tablet replica
I20260929 03:01:09.648092 22428 raft_consensus.cc:2229] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:01:09.648782 22428 raft_consensus.cc:2258] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:01:09.649559 22427 tablet_replica.cc:330] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a: stopping tablet replica
I20260929 03:01:09.649931 22427 raft_consensus.cc:2229] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:01:09.650559 22427 raft_consensus.cc:2258] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:01:09.652726 22427 ts_tablet_manager.cc:1905] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:01:09.653501 22429 ts_tablet_manager.cc:1905] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:01:09.658416 22428 ts_tablet_manager.cc:1905] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:01:09.660393 22429 ts_tablet_manager.cc:1918] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:01:09.660626 22429 log.cc:1196] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/23c15a8cfeff439f807f8f39bf0e8860
I20260929 03:01:09.661281 22429 ts_tablet_manager.cc:1939] T 23c15a8cfeff439f807f8f39bf0e8860 P 145189612cb14344957d9cc15df55a5f: Deleting consensus metadata
I20260929 03:01:09.663136 21887 catalog_manager.cc:4841] TS 145189612cb14344957d9cc15df55a5f (127.8.208.66:41843): tablet 23c15a8cfeff439f807f8f39bf0e8860 (table test-table [id=8c39b053254245cf99ce7bae29c09629]) successfully deleted
I20260929 03:01:09.665911 22427 ts_tablet_manager.cc:1918] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:01:09.666103 22427 log.cc:1196] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/23c15a8cfeff439f807f8f39bf0e8860
I20260929 03:01:09.666615 22427 ts_tablet_manager.cc:1939] T 23c15a8cfeff439f807f8f39bf0e8860 P 01044b64599d4e1e9492dac347f2090a: Deleting consensus metadata
I20260929 03:01:09.668428 21885 catalog_manager.cc:4841] TS 01044b64599d4e1e9492dac347f2090a (127.8.208.65:37811): tablet 23c15a8cfeff439f807f8f39bf0e8860 (table test-table [id=8c39b053254245cf99ce7bae29c09629]) successfully deleted
I20260929 03:01:09.669543 22428 ts_tablet_manager.cc:1918] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:01:09.669719 22428 log.cc:1196] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/23c15a8cfeff439f807f8f39bf0e8860
I20260929 03:01:09.670265 22392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.642681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38564 (local address 127.8.208.65:37811)
0929 03:01:09.642906 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:09.642915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:09.648551 (+ 5636us) server_negotiation.cc:407] Connection header received
0929 03:01:09.650356 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:09.650362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:09.650466 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:09.650562 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:09.655019 (+ 4457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.656206 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.658210 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.658824 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.658927 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.662598 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:09.662611 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:09.662629 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:09.662674 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:09.666809 (+ 4135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:09.667657 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:09.667666 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:09.667669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:09.667833 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:09.668322 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:09.668327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:09.668330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:09.668848 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:09.669070 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:09.669648 (+ 578us) server_negotiation.cc:299] Negotiation successful
0929 03:01:09.669869 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 03:01:09.670301 22428 ts_tablet_manager.cc:1939] T 23c15a8cfeff439f807f8f39bf0e8860 P bd77967ad2484e2db1fe60745e667c84: Deleting consensus metadata
I20260929 03:01:09.670868 22092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:09.672021 21887 catalog_manager.cc:4841] TS bd77967ad2484e2db1fe60745e667c84 (127.8.208.67:42343): tablet 23c15a8cfeff439f807f8f39bf0e8860 (table test-table [id=8c39b053254245cf99ce7bae29c09629]) successfully deleted
I20260929 03:01:09.762847 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 21937
I20260929 03:01:09.785845 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 22101
I20260929 03:01:09.799743 22395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.786691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43950 (local address 127.8.208.66:41843)
0929 03:01:09.786895 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:09.786900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:09.786920 (+ 20us) server_negotiation.cc:407] Connection header received
0929 03:01:09.787091 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:09.787096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:09.787173 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:09.787282 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:09.788560 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.789334 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.790562 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.790970 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.791106 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.794077 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:09.794093 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:09.794108 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:09.794143 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:09.796852 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:09.797600 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:09.797610 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:09.797613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:09.797759 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:09.798124 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:09.798130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:09.798133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:09.798456 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:09.798744 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:09.799079 (+ 335us) server_negotiation.cc:299] Negotiation successful
0929 03:01:09.799322 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 03:01:09.800330 22236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:09.899602 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 22101
I20260929 03:01:09.917572 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 22245
I20260929 03:01:09.935469 22394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:09.918434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46492 (local address 127.8.208.67:42343)
0929 03:01:09.918634 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:09.918640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:09.918662 (+ 22us) server_negotiation.cc:407] Connection header received
0929 03:01:09.918769 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:09.918777 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:09.918862 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:09.918987 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:09.920661 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.922001 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.923798 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:09.924480 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:09.924619 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:09.927870 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:09.927885 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:09.927906 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:09.927957 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:09.932234 (+ 4277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:09.933009 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:09.933029 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:09.933032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:09.933266 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:09.933723 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:09.933728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:09.933732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:09.934170 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:09.934478 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:09.934807 (+ 329us) server_negotiation.cc:299] Negotiation successful
0929 03:01:09.935030 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 03:01:09.936223 22370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:10.060207 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 22245
I20260929 03:01:10.079679 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 21867
I20260929 03:01:10.080894 21930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:10.194761 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 21867
I20260929 03:01:10.216751 9025 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 03:01:10.551136 9025 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 03:01:10.551370 9025 mini_ranger.cc:66] Stopping Ranger...
I20260929 03:01:10.896229 9025 mini_ranger.cc:68] Stopped Ranger
2026-09-29 03:01:10.896 UTC [20918] LOG: received smart shutdown request
2026-09-29 03:01:10.899 UTC [20918] LOG: background worker "logical replication launcher" (PID 20927) exited with exit code 1
2026-09-29 03:01:10.900 UTC [20922] LOG: shutting down
2026-09-29 03:01:11.060 UTC [20918] LOG: database system is shut down
2026-09-29T03:01:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52640 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:01:11 dist-test-slave-m1rv krb5kdc[22443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:01:11 dist-test-slave-m1rv krb5kdc[22443](info): set up 2 sockets
Sep 29 03:01:11 dist-test-slave-m1rv krb5kdc[22443](info): commencing operation
krb5kdc: starting...
W20260929 03:01:12.305892 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.008s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:01:12 dist-test-slave-m1rv krb5kdc[22443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:01:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:01:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:01:12.528409 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:45723
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:45723,127.8.208.125:35647,127.8.208.124:37589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:12.852079 22456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:12.852411 22456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:12.852578 22456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:12.859879 22456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:01:12.860021 22456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:12.860117 22456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:12.860226 22456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:01:12.860314 22456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:01:12.870414 22456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:45723,127.8.208.125:35647,127.8.208.124:37589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:45723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:12.871861 22456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:12.873250 22456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:12.885221 22462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:12.885517 22464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:12.885553 22461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:12.886502 22456 server_base.cc:1034] running on GCE node
I20260929 03:01:12.887364 22456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:12.888728 22456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:12.889966 22456 hybrid_clock.cc:648] HybridClock initialized: now 1790650872889902 us; error 46 us; skew 500 ppm
I20260929 03:01:12.890352 22456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:12 dist-test-slave-m1rv krb5kdc[22443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650872, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:12.893574 22456 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:01:12.895021 22456 webserver.cc:460] Webserver started at http://127.8.208.126:37889/ using document root <none> and password file <none>
I20260929 03:01:12.895700 22456 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:12.895838 22456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:12.896134 22456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:01:12.899073 22456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "df7d3458732d477995671cdbb9104302"
format_stamp: "Formatted at 2026-09-29 03:01:12 on dist-test-slave-m1rv"
server_key: "28f35e7634ab291d140c3aa86f12d8fd"
server_key_iv: "f2f97014ccf81a6e7f98f440bc1908f1"
server_key_version: "encryptionkey@0"
I20260929 03:01:12.899886 22456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "df7d3458732d477995671cdbb9104302"
format_stamp: "Formatted at 2026-09-29 03:01:12 on dist-test-slave-m1rv"
server_key: "28f35e7634ab291d140c3aa86f12d8fd"
server_key_iv: "f2f97014ccf81a6e7f98f440bc1908f1"
server_key_version: "encryptionkey@0"
I20260929 03:01:12.904919 22456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 03:01:12.908324 22471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:12.909636 22456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:01:12.909802 22456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "df7d3458732d477995671cdbb9104302"
format_stamp: "Formatted at 2026-09-29 03:01:12 on dist-test-slave-m1rv"
server_key: "28f35e7634ab291d140c3aa86f12d8fd"
server_key_iv: "f2f97014ccf81a6e7f98f440bc1908f1"
server_key_version: "encryptionkey@0"
I20260929 03:01:12.910027 22456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:12.930980 22456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:12.935106 22456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:12.935379 22456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:12.953408 22456 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:45723
I20260929 03:01:12.953440 22522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:45723 every 8 connection(s)
I20260929 03:01:12.954684 22456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:01:12.957973 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 22456
I20260929 03:01:12.958199 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:01:12.958257 22523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:12.958865 9025 external_mini_cluster.cc:1428] Setting key 02d9745c1e8103373e2610824538f2d7
I20260929 03:01:12.964468 22523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.208.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.125@KRBTEST.COM" created.
Sep 29 03:01:12 dist-test-slave-m1rv krb5kdc[22443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650872, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:12.989579 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:12.968775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34642 (local address 127.8.208.126:45723)
0929 03:01:12.969500 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:12.969508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:12.969563 (+ 55us) server_negotiation.cc:407] Connection header received
0929 03:01:12.970608 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:12.970614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:12.970819 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:12.971019 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:12.972178 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:12.973416 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:12.974747 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:12.975283 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:12.975467 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:12.981415 (+ 5948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:12.981422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:12.981434 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:12.981462 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:12.984968 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:12.985835 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:12.985846 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:12.985852 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:12.986068 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:12.986501 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:12.986509 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:12.986511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:12.987045 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:12.987262 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:12.987772 (+ 510us) server_negotiation.cc:299] Negotiation successful
0929 03:01:12.988109 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":533,"thread_start_us":373,"threads_started":1}
I20260929 03:01:12.989610 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:12.967487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:45723 (local address 127.0.0.1:34642)
0929 03:01:12.968197 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:12.968233 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:12.970259 (+ 2026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:12.971033 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:12.971040 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:12.971696 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:12.972001 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:12.972016 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:12.973562 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:12.973567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:12.974613 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:12.974624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:12.975491 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:12.975506 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:12.975693 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:12.976322 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:12.976343 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:12.981164 (+ 4821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:12.985127 (+ 3963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:12.985141 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:12.985154 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:12.985684 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:12.986186 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:12.986193 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:12.986198 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:12.986371 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:12.987187 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:12.987196 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:12.987404 (+ 208us) client_negotiation.cc:769] Sending connection context
0929 03:01:12.987649 (+ 245us) client_negotiation.cc:240] Negotiation successful
0929 03:01:12.988219 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":143,"thread_start_us":305,"threads_started":1}
I20260929 03:01:12.992955 22523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:01:12.994084 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:12.993709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45628)
0929 03:01:12.993871 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:12.993974 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260929 03:01:12.994339 22473 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
I20260929 03:01:12.996001 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:12.995592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45636)
0929 03:01:12.995753 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:12.995851 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 03:01:12.996192 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:35647: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.125.keytab.
Entry for principal kudu/127.8.208.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.125.keytab.
I20260929 03:01:13.006817 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.125:35647
--webserver_interface=127.8.208.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.125.keytab
--principal=kudu/127.8.208.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:45723,127.8.208.125:35647,127.8.208.124:37589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:01:13.045526 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } attempt: 1
I20260929 03:01:13.046761 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.046281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45652)
0929 03:01:13.046476 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.046598 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 03:01:13.047688 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.047357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45658)
0929 03:01:13.047495 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.047572 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 03:01:13.047865 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:35647: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
I20260929 03:01:13.116084 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } attempt: 2
I20260929 03:01:13.117252 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.116719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45666)
0929 03:01:13.116959 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.117054 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 03:01:13.118171 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.117827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45676)
0929 03:01:13.117962 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.118047 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 03:01:13.118371 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:35647: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
I20260929 03:01:13.209649 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } attempt: 3
I20260929 03:01:13.210785 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.210344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45678)
0929 03:01:13.210551 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.210657 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 03:01:13.211796 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.211439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45682)
0929 03:01:13.211602 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.211672 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 03:01:13.211990 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:35647: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
W20260929 03:01:13.328436 22530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:13.328771 22530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:13.328943 22530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:13.336112 22530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:01:13.336238 22530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:13.336335 22530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:13.336447 22530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:01:13.336531 22530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:01:13.346168 22530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:45723,127.8.208.125:35647,127.8.208.124:37589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.125.keytab
--principal=kudu/127.8.208.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.125:35647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.208.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:13.347532 22530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:13.348788 22530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:01:13.355219 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } attempt: 4
I20260929 03:01:13.356359 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.355785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45698)
0929 03:01:13.356077 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.356203 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260929 03:01:13.357463 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.357028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45710)
0929 03:01:13.357210 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.357315 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 03:01:13.357663 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:35647: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35647: connect: Connection refused (error 111)
W20260929 03:01:13.360827 22535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:13.360953 22536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:13.360903 22538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:13.361626 22530 server_base.cc:1034] running on GCE node
I20260929 03:01:13.362335 22530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:13.363637 22530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:13.364853 22530 hybrid_clock.cc:648] HybridClock initialized: now 1790650873364815 us; error 51 us; skew 500 ppm
I20260929 03:01:13.365232 22530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:13 dist-test-slave-m1rv krb5kdc[22443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650873, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:13.376821 22530 init.cc:377] Logged in from keytab as kudu/127.8.208.125@KRBTEST.COM (short username kudu)
I20260929 03:01:13.378399 22530 webserver.cc:460] Webserver started at http://127.8.208.125:33145/ using document root <none> and password file <none>
I20260929 03:01:13.379083 22530 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:13.379225 22530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:13.379516 22530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:01:13.382460 22530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/data/instance:
uuid: "612830e1615e467cb6c8e1256e63e4ed"
format_stamp: "Formatted at 2026-09-29 03:01:13 on dist-test-slave-m1rv"
server_key: "65e971fb89e1324b13d2a39fb20c19b2"
server_key_iv: "317fd5f21dab2e1ce2fc4b0cdd77f3c8"
server_key_version: "encryptionkey@0"
I20260929 03:01:13.383256 22530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/wal/instance:
uuid: "612830e1615e467cb6c8e1256e63e4ed"
format_stamp: "Formatted at 2026-09-29 03:01:13 on dist-test-slave-m1rv"
server_key: "65e971fb89e1324b13d2a39fb20c19b2"
server_key_iv: "317fd5f21dab2e1ce2fc4b0cdd77f3c8"
server_key_version: "encryptionkey@0"
I20260929 03:01:13.388171 22530 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 03:01:13.391345 22545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:13.392632 22530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 03:01:13.392804 22530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/wal
uuid: "612830e1615e467cb6c8e1256e63e4ed"
format_stamp: "Formatted at 2026-09-29 03:01:13 on dist-test-slave-m1rv"
server_key: "65e971fb89e1324b13d2a39fb20c19b2"
server_key_iv: "317fd5f21dab2e1ce2fc4b0cdd77f3c8"
server_key_version: "encryptionkey@0"
I20260929 03:01:13.392985 22530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:13.411388 22530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:13.415468 22530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:13.415753 22530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:13.433072 22530 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.125:35647
I20260929 03:01:13.433118 22596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.125:35647 every 8 connection(s)
I20260929 03:01:13.434517 22530 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
I20260929 03:01:13.437692 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 22530
I20260929 03:01:13.437885 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-1/wal/instance
I20260929 03:01:13.438422 9025 external_mini_cluster.cc:1428] Setting key 4fc35bd1a3cb186139f889b598263398
I20260929 03:01:13.438460 22597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:13.447933 22597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.208.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.124@KRBTEST.COM" created.
Sep 29 03:01:13 dist-test-slave-m1rv krb5kdc[22443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650873, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.125@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:13.470249 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.450508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34658 (local address 127.8.208.126:45723)
0929 03:01:13.450763 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:13.450772 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:13.452629 (+ 1857us) server_negotiation.cc:407] Connection header received
0929 03:01:13.454522 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:13.454540 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:13.454636 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:13.454756 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:13.456265 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.457558 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.459024 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.459740 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.459881 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.463850 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:13.463855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:13.463858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:13.463892 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:13.466655 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.467630 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.467634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.467636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.467754 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.468269 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.468272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.468274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.468547 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:13.468656 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:13.469577 (+ 921us) server_negotiation.cc:299] Negotiation successful
0929 03:01:13.469841 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 03:01:13.470953 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.450984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:45723 (local address 127.0.0.1:34658)
0929 03:01:13.452454 (+ 1470us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.452497 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:13.454264 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:13.454953 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:13.454960 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:13.455454 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:13.456090 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:13.456107 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.457702 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.457705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:13.458873 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:13.458884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.459949 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.459953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:13.460130 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.460761 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:13.460781 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:13.463608 (+ 2827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:13.466832 (+ 3224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:13.466844 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:13.466864 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:13.467514 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:13.467891 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:13.467900 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:13.467906 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:13.468159 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:13.468678 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:13.468689 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:13.469136 (+ 447us) client_negotiation.cc:769] Sending connection context
0929 03:01:13.469441 (+ 305us) client_negotiation.cc:240] Negotiation successful
0929 03:01:13.469850 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1188,"thread_start_us":976,"threads_started":1}
I20260929 03:01:13.472649 22597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:01:13 dist-test-slave-m1rv krb5kdc[22443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650873, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.125@KRBTEST.COM for kudu/127.8.208.125@KRBTEST.COM
I20260929 03:01:13.489822 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.473471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45714)
0929 03:01:13.473647 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.473662 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:13.473741 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:13.475157 (+ 1416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:13.475165 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:13.475774 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:13.475995 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:13.476009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.477511 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.477519 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:13.479169 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:13.479188 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.480263 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.480270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:13.480561 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.481233 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:13.481261 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:13.483855 (+ 2594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:13.487166 (+ 3311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:13.487172 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:13.487176 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:13.487650 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:13.488052 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:13.488059 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:13.488063 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:13.488200 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:13.488808 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:13.488813 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:13.488959 (+ 146us) client_negotiation.cc:769] Sending connection context
0929 03:01:13.489108 (+ 149us) client_negotiation.cc:240] Negotiation successful
0929 03:01:13.489408 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 03:01:13.489984 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.473771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45714 (local address 127.8.208.125:35647)
0929 03:01:13.474502 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:13.474534 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:13.474567 (+ 33us) server_negotiation.cc:407] Connection header received
0929 03:01:13.474628 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:13.474638 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:13.474916 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:13.475123 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:13.476236 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.477375 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.479428 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.480124 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.480228 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.484098 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:13.484110 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:13.484125 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:13.484156 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:13.487036 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.487810 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.487817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.487820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.487940 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.488330 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.488334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.488336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.488682 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:13.488935 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:13.489280 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 03:01:13.489538 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":375,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.124.keytab.
Entry for principal kudu/127.8.208.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.124.keytab.
I20260929 03:01:13.492561 22597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:01:13.492864 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.124:37589
--webserver_interface=127.8.208.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.124.keytab
--principal=kudu/127.8.208.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:45723,127.8.208.125:35647,127.8.208.124:37589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:01:13.493903 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.493467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51636)
0929 03:01:13.493624 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.493762 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260929 03:01:13.494277 22549 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
I20260929 03:01:13.495832 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.495477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51650)
0929 03:01:13.495637 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.495733 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 03:01:13.496124 22597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:37589: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
I20260929 03:01:13.545483 22597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } attempt: 1
I20260929 03:01:13.546705 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.546245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51664)
0929 03:01:13.546438 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.546558 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260929 03:01:13.547703 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.547405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51666)
0929 03:01:13.547547 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.547607 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 03:01:13.547962 22597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:37589: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
I20260929 03:01:13.616267 22597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } attempt: 2
I20260929 03:01:13.617415 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.616968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51670)
0929 03:01:13.617157 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.617265 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260929 03:01:13.618326 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.618004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51678)
0929 03:01:13.618148 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.618215 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 03:01:13.618562 22597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:37589: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
I20260929 03:01:13.656934 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } attempt: 5
Sep 29 03:01:13 dist-test-slave-m1rv krb5kdc[22443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650872, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for kudu/127.8.208.125@KRBTEST.COM
I20260929 03:01:13.672235 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.657655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45728)
0929 03:01:13.657909 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.657935 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:13.658106 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:13.658473 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:13.658478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:13.659088 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:13.659353 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:13.659366 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.660310 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.660317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:13.662040 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:13.662059 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.662812 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.662819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:13.663076 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.663791 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:13.663818 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:13.666212 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:13.669934 (+ 3722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:13.669938 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:13.669940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:13.670303 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:13.670768 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:13.670772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:13.670774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:13.670903 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:13.671555 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:13.671558 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:13.671677 (+ 119us) client_negotiation.cc:769] Sending connection context
0929 03:01:13.671789 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 03:01:13.671982 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 03:01:13.672453 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.657803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45728 (local address 127.8.208.125:35647)
0929 03:01:13.657957 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:13.657961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:13.658031 (+ 70us) server_negotiation.cc:407] Connection header received
0929 03:01:13.658236 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:13.658240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:13.658321 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:13.658403 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:13.659495 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.660163 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.662199 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.662643 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.662736 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.666448 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:13.666455 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:13.666459 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:13.666499 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:13.669802 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.670436 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.670444 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.670449 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.670633 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.671019 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.671026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.671030 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.671422 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:13.671607 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:13.671911 (+ 304us) server_negotiation.cc:299] Negotiation successful
0929 03:01:13.672145 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:01:13.673488 22523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:01:13.674749 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.674189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51684)
0929 03:01:13.674411 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.674543 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260929 03:01:13.675776 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.675374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51700)
0929 03:01:13.675542 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.675647 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 03:01:13.676034 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:37589: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
I20260929 03:01:13.709756 22597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } attempt: 3
I20260929 03:01:13.710866 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.710389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51714)
0929 03:01:13.710623 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.710715 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 03:01:13.711970 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.711589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51730)
0929 03:01:13.711798 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.711870 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260929 03:01:13.712217 22597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:37589: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
I20260929 03:01:13.727257 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } attempt: 1
I20260929 03:01:13.728477 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.727845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51738)
0929 03:01:13.728104 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.728258 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260929 03:01:13.729594 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.729146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51746)
0929 03:01:13.729346 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.729446 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260929 03:01:13.729842 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:37589: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
I20260929 03:01:13.798058 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } attempt: 2
I20260929 03:01:13.799300 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.798665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51762)
0929 03:01:13.798942 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.799066 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260929 03:01:13.800349 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.799940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51768)
0929 03:01:13.800107 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.800208 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 03:01:13.800616 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:37589: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
W20260929 03:01:13.823004 22604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:13.823307 22604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:13.823469 22604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:13.830564 22604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:01:13.830670 22604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:13.830718 22604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:13.830806 22604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:01:13.830892 22604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:01:13.840698 22604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:45723,127.8.208.125:35647,127.8.208.124:37589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.124.keytab
--principal=kudu/127.8.208.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.124:37589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.208.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:13.842006 22604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:13.843319 22604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:13.854710 22610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:13.854791 22609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:13.854769 22612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:13.855393 22604 server_base.cc:1034] running on GCE node
I20260929 03:01:13.855424 22597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } attempt: 4
I20260929 03:01:13.856297 22604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:13.856456 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.856036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51776)
0929 03:01:13.856238 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.856313 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 03:01:13.857411 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.857071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51790)
0929 03:01:13.857241 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.857314 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 03:01:13.857622 22597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:37589: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
I20260929 03:01:13.857935 22604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:13.859148 22604 hybrid_clock.cc:648] HybridClock initialized: now 1790650873859102 us; error 37 us; skew 500 ppm
I20260929 03:01:13.859478 22604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:13 dist-test-slave-m1rv krb5kdc[22443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650873, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:13.862445 22604 init.cc:377] Logged in from keytab as kudu/127.8.208.124@KRBTEST.COM (short username kudu)
I20260929 03:01:13.863838 22604 webserver.cc:460] Webserver started at http://127.8.208.124:40961/ using document root <none> and password file <none>
I20260929 03:01:13.864513 22604 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:13.864655 22604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:13.864939 22604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:01:13.867810 22604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/data/instance:
uuid: "c59e5dd4f4514345b409abb7f51e8cfe"
format_stamp: "Formatted at 2026-09-29 03:01:13 on dist-test-slave-m1rv"
server_key: "d43dfc75d9cdd4778c28f28d4f8b97bc"
server_key_iv: "d7d7f1d24c3f2ca75afce5fd2f599cd2"
server_key_version: "encryptionkey@0"
I20260929 03:01:13.868616 22604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/wal/instance:
uuid: "c59e5dd4f4514345b409abb7f51e8cfe"
format_stamp: "Formatted at 2026-09-29 03:01:13 on dist-test-slave-m1rv"
server_key: "d43dfc75d9cdd4778c28f28d4f8b97bc"
server_key_iv: "d7d7f1d24c3f2ca75afce5fd2f599cd2"
server_key_version: "encryptionkey@0"
I20260929 03:01:13.874070 22604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 03:01:13.878593 22619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:13.879856 22604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260929 03:01:13.880025 22604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/wal
uuid: "c59e5dd4f4514345b409abb7f51e8cfe"
format_stamp: "Formatted at 2026-09-29 03:01:13 on dist-test-slave-m1rv"
server_key: "d43dfc75d9cdd4778c28f28d4f8b97bc"
server_key_iv: "d7d7f1d24c3f2ca75afce5fd2f599cd2"
server_key_version: "encryptionkey@0"
I20260929 03:01:13.880229 22604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:13.892187 22604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:13.895911 22604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:13.896216 22604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:13.906811 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } attempt: 3
I20260929 03:01:13.908023 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.907387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51796)
0929 03:01:13.907678 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.907798 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260929 03:01:13.909144 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.908657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51804)
0929 03:01:13.908894 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.909006 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260929 03:01:13.909482 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:37589: Network error: Client connection negotiation failed: client connection to 127.8.208.124:37589: connect: Connection refused (error 111)
I20260929 03:01:13.915047 22604 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.124:37589
I20260929 03:01:13.915074 22670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.124:37589 every 8 connection(s)
I20260929 03:01:13.916383 22604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
I20260929 03:01:13.919915 22671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:13.924801 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 22604
I20260929 03:01:13.925002 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790650672582614-9025-0/minicluster-data/master-2/wal/instance
I20260929 03:01:13.925541 9025 external_mini_cluster.cc:1428] Setting key fe17d65ff3e7fe5da602d8a765a1bd96
I20260929 03:01:13.926456 22671 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:01:13 dist-test-slave-m1rv krb5kdc[22443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650873, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.124@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
Sep 29 03:01:13 dist-test-slave-m1rv krb5kdc[22443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.124@KRBTEST.COM
I20260929 03:01:13.945506 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.929273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34662 (local address 127.8.208.126:45723)
0929 03:01:13.929485 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:13.929493 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:13.929998 (+ 505us) server_negotiation.cc:407] Connection header received
0929 03:01:13.930602 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:13.930607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:13.930690 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:13.930804 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:13.932117 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.932852 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.934624 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.935285 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.935376 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.939394 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:13.939403 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:13.939409 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:13.939453 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:13.942322 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.943134 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.943139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.943143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.943274 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.943703 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.943708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.943711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.944079 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:13.944248 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:13.944983 (+ 735us) server_negotiation.cc:299] Negotiation successful
0929 03:01:13.945240 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:01:13.946619 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.929258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:45723 (local address 127.0.0.1:34662)
0929 03:01:13.929864 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.929898 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:13.930373 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:13.930893 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:13.930905 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:13.931570 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:13.931949 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:13.931966 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.933025 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.933030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:13.934470 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:13.934482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.935494 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.935503 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:13.935858 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.936641 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:13.936667 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:13.939203 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:13.942456 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:13.942469 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:13.942480 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:13.943007 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:13.943383 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:13.943388 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:13.943392 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:13.943586 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:13.944207 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:13.944213 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:13.944552 (+ 339us) client_negotiation.cc:769] Sending connection context
0929 03:01:13.944839 (+ 287us) client_negotiation.cc:240] Negotiation successful
0929 03:01:13.945280 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":34,"thread_start_us":275,"threads_started":1}
I20260929 03:01:13.946801 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.927292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51814 (local address 127.8.208.124:37589)
0929 03:01:13.928050 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:13.928061 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:13.928102 (+ 41us) server_negotiation.cc:407] Connection header received
0929 03:01:13.930364 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:13.930383 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:13.930607 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:13.930789 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:13.932157 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.933630 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.935195 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.936013 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.936166 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.940856 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:13.940863 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:13.940874 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:13.940903 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:13.943677 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.944431 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.944438 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.944441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.944645 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.945060 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.945064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.945067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.945472 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:13.945999 (+ 527us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:13.946159 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 03:01:13.946377 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":561,"spinlock_wait_cycles":17664,"thread_start_us":363,"threads_started":1}
I20260929 03:01:13.948895 22671 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:01:13 dist-test-slave-m1rv krb5kdc[22443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650873, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.124@KRBTEST.COM for kudu/127.8.208.125@KRBTEST.COM
Sep 29 03:01:13 dist-test-slave-m1rv krb5kdc[22443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:13.962930 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.949700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35647 (local address 127.0.0.1:45740)
0929 03:01:13.949913 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.949930 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:13.950064 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:13.950360 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:13.950364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:13.950713 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:13.950854 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:13.950862 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.951802 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.951809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:13.952710 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:13.952723 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.953750 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.953756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:13.953928 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.954468 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:13.954491 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:13.956976 (+ 2485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:13.960535 (+ 3559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:13.960540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:13.960544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:13.961006 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:13.961440 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:13.961445 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:13.961449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:13.961595 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:13.962067 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:13.962072 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:13.962208 (+ 136us) client_negotiation.cc:769] Sending connection context
0929 03:01:13.962337 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 03:01:13.962583 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260929 03:01:13.963233 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.949699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45740 (local address 127.8.208.125:35647)
0929 03:01:13.949885 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:13.949891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:13.950002 (+ 111us) server_negotiation.cc:407] Connection header received
0929 03:01:13.950173 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:13.950179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:13.950244 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:13.950338 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:13.950993 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.951672 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.952875 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.953563 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.953687 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.957243 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:13.957251 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:13.957256 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:13.957297 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:13.960404 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.961120 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.961128 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.961134 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.961321 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.961697 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.961704 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.961708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.961945 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:13.962088 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:13.962436 (+ 348us) server_negotiation.cc:299] Negotiation successful
0929 03:01:13.962649 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 03:01:13.964193 22671 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:01:13.966289 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.949651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.8.208.126:45723)
0929 03:01:13.949818 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:13.949822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:13.950033 (+ 211us) server_negotiation.cc:407] Connection header received
0929 03:01:13.950193 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:13.950198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:13.950255 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:13.950348 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:13.951429 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.952132 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.953667 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.954330 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.954441 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.959441 (+ 5000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:13.959447 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:13.959451 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:13.959489 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:13.962865 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.963837 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.963842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.963846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.964004 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.964875 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.964882 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.964886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.965275 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:13.965541 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:13.965784 (+ 243us) server_negotiation.cc:299] Negotiation successful
0929 03:01:13.965992 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Sep 29 03:01:13 dist-test-slave-m1rv krb5kdc[22443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650873, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.124@KRBTEST.COM for kudu/127.8.208.124@KRBTEST.COM
Sep 29 03:01:13 dist-test-slave-m1rv krb5kdc[22443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.125@KRBTEST.COM
I20260929 03:01:13.978080 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.964963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51824)
0929 03:01:13.965162 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:13.965222 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:13.965317 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:13.965584 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:13.965588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:13.965950 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:13.966069 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:13.966077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.967467 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.967471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:13.968446 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:13.968458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.969442 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.969446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:13.969628 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.970156 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:13.970176 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:13.972634 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:13.975548 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:13.975556 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:13.975559 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:13.976107 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:13.976463 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:13.976469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:13.976474 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:13.976660 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:13.977290 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:13.977293 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:13.977457 (+ 164us) client_negotiation.cc:769] Sending connection context
0929 03:01:13.977592 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 03:01:13.977802 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":24}
I20260929 03:01:13.978376 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.964908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51824 (local address 127.8.208.124:37589)
0929 03:01:13.965080 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:13.965086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:13.965275 (+ 189us) server_negotiation.cc:407] Connection header received
0929 03:01:13.965417 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:13.965426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:13.965473 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:13.965563 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:13.966357 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.967350 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.968574 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.969309 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.969445 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.972813 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:13.972821 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:13.972825 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:13.972860 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:13.975417 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.976232 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.976236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.976238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.976352 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.976772 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.976779 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.976785 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.977152 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:13.977327 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:13.977724 (+ 397us) server_negotiation.cc:299] Negotiation successful
0929 03:01:13.977946 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":55}
I20260929 03:01:13.984167 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.967800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.8.208.125:35647)
0929 03:01:13.967947 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:13.967954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:13.967972 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:01:13.968158 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:13.968163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:13.968227 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:13.968320 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:13.969944 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.971026 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.972845 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:13.973568 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:13.973696 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:13.978410 (+ 4714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:13.978419 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:13.978425 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:13.978463 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:13.981673 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.982378 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.982381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.982383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.982508 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:13.982903 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:13.982906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:13.982908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:13.983141 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:13.983393 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:13.983731 (+ 338us) server_negotiation.cc:299] Negotiation successful
0929 03:01:13.983900 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 03:01:13.984181 22671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe: Bootstrap starting.
I20260929 03:01:13.987798 22671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:13.988714 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:01:13.988909 22671 log.cc:824] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:13.991626 22671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe: No bootstrap required, opened a new log
I20260929 03:01:14.005405 22671 raft_consensus.cc:348] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } }
I20260929 03:01:14.005954 22671 raft_consensus.cc:374] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:14.006093 22671 raft_consensus.cc:729] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59e5dd4f4514345b409abb7f51e8cfe, State: Initialized, Role: FOLLOWER
I20260929 03:01:14.007701 22671 consensus_queue.cc:260] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } }
I20260929 03:01:14.021787 22684 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } } }
I20260929 03:01:14.022272 22684 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:01:14.023568 22671 sys_catalog.cc:564] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:01:14.025389 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.994234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45764 (local address 127.8.208.125:35647)
0929 03:01:13.999738 (+ 5504us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:13.999746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:14.007071 (+ 7325us) server_negotiation.cc:407] Connection header received
0929 03:01:14.007251 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:14.007257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:14.007328 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:14.007394 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:14.009816 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.010947 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.013265 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.013906 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.014022 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:14.018146 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:14.018153 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:14.018158 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:14.018202 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:14.021908 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:14.022705 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:14.022710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:14.022714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:14.022928 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:14.023423 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:14.023429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:14.023432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:14.023807 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:14.024102 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:14.024734 (+ 632us) server_negotiation.cc:299] Negotiation successful
0929 03:01:14.024969 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5342}
I20260929 03:01:14.026576 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.994212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.8.208.126:45723)
0929 03:01:13.994373 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:13.994379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:14.006402 (+ 12023us) server_negotiation.cc:407] Connection header received
0929 03:01:14.007078 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:14.007085 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:14.007175 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:14.007261 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:14.010810 (+ 3549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.011912 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.014119 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.014800 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.014900 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:14.018678 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:14.018685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:14.018690 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:14.018731 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:14.023029 (+ 4298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:14.024011 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:14.024017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:14.024022 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:14.024189 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:14.024680 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:14.024685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:14.024688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:14.025037 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:14.025318 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:14.025990 (+ 672us) server_negotiation.cc:299] Negotiation successful
0929 03:01:14.026228 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 03:01:14.033648 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:13.993785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51828 (local address 127.8.208.124:37589)
0929 03:01:13.993954 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:13.993962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:14.010651 (+ 16689us) server_negotiation.cc:407] Connection header received
0929 03:01:14.010769 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:14.010777 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:14.010899 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:14.011032 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:14.014498 (+ 3466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.015977 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.019043 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.020019 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.020141 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:14.026025 (+ 5884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:14.026033 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:14.026038 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:14.026107 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:14.030774 (+ 4667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:14.031586 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:14.031591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:14.031600 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:14.031754 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:14.032146 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:14.032155 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:14.032159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:14.032620 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:14.032882 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:14.033261 (+ 379us) server_negotiation.cc:299] Negotiation successful
0929 03:01:14.033406 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260929 03:01:14.039456 22695 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:01:14.039608 22695 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:01:14.034493 22604 master_runner.cc:186] Error getting master registration for 127.8.208.126:45723: OK, instance_id { permanent_uuid: "df7d3458732d477995671cdbb9104302" instance_seqno: 1790650872948276 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 03:01:14.043574 22604 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 03:01:14.086696 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } attempt: 4
Sep 29 03:01:14 dist-test-slave-m1rv krb5kdc[22443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650872, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for kudu/127.8.208.124@KRBTEST.COM
I20260929 03:01:14.098331 22684 raft_consensus.cc:482] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:01:14.098662 22684 raft_consensus.cc:504] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } }
I20260929 03:01:14.100507 22684 leader_election.cc:288] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df7d3458732d477995671cdbb9104302 (127.8.208.126:45723), 612830e1615e467cb6c8e1256e63e4ed (127.8.208.125:35647)
I20260929 03:01:14.103477 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:14.087311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51840)
0929 03:01:14.087509 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:14.087526 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:14.087623 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:14.087971 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:14.087974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:14.088354 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:14.088513 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:14.088521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.089974 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.089978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:14.090975 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:14.090986 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.091974 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.091978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:14.092108 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:14.092742 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:14.092758 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:14.095065 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:14.100432 (+ 5367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:14.100438 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:14.100442 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:14.101005 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:14.101560 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:14.101566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:14.101588 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:14.101760 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:14.102410 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:14.102415 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:14.102614 (+ 199us) client_negotiation.cc:769] Sending connection context
0929 03:01:14.102768 (+ 154us) client_negotiation.cc:240] Negotiation successful
0929 03:01:14.103060 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 03:01:14.103544 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:14.087387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51840 (local address 127.8.208.124:37589)
0929 03:01:14.087589 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:14.087595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:14.087614 (+ 19us) server_negotiation.cc:407] Connection header received
0929 03:01:14.087712 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:14.087718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:14.087840 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:14.087936 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:14.088659 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.089854 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.091122 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.091848 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.091969 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:14.095252 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:14.095261 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:14.095271 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:14.095311 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:14.100272 (+ 4961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:14.101206 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:14.101213 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:14.101219 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:14.101391 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:14.101897 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:14.101902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:14.101905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:14.102273 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:14.102403 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:14.102898 (+ 495us) server_negotiation.cc:299] Negotiation successful
0929 03:01:14.103154 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 03:01:14.109886 22523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302: Bootstrap starting.
I20260929 03:01:14.114799 22523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:14.116175 22523 log.cc:824] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:14.119378 22523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302: No bootstrap required, opened a new log
I20260929 03:01:14.131523 22523 raft_consensus.cc:348] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } }
I20260929 03:01:14.131947 22523 raft_consensus.cc:374] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:14.132082 22523 raft_consensus.cc:729] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df7d3458732d477995671cdbb9104302, State: Initialized, Role: FOLLOWER
I20260929 03:01:14.132766 22523 consensus_queue.cc:260] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } }
I20260929 03:01:14.133842 22698 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } } }
I20260929 03:01:14.134262 22698 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:01:14.135254 22523 sys_catalog.cc:564] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:01:14.135277 22498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df7d3458732d477995671cdbb9104302" is_pre_election: true
I20260929 03:01:14.135869 22498 raft_consensus.cc:2454] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c59e5dd4f4514345b409abb7f51e8cfe in term 0.
I20260929 03:01:14.136601 22620 leader_election.cc:302] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c59e5dd4f4514345b409abb7f51e8cfe, df7d3458732d477995671cdbb9104302; no voters:
I20260929 03:01:14.137113 22684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:01:14.137300 22684 raft_consensus.cc:482] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:01:14.137416 22684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:14.140766 22684 raft_consensus.cc:504] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } }
I20260929 03:01:14.141896 22684 leader_election.cc:288] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [CANDIDATE]: Term 1 election: Requested vote from peers df7d3458732d477995671cdbb9104302 (127.8.208.126:45723), 612830e1615e467cb6c8e1256e63e4ed (127.8.208.125:35647)
I20260929 03:01:14.143479 22498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df7d3458732d477995671cdbb9104302"
I20260929 03:01:14.143711 22498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:14.147080 22498 raft_consensus.cc:2454] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c59e5dd4f4514345b409abb7f51e8cfe in term 1.
I20260929 03:01:14.147535 22620 leader_election.cc:302] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c59e5dd4f4514345b409abb7f51e8cfe, df7d3458732d477995671cdbb9104302; no voters:
I20260929 03:01:14.147833 22684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [term 1 FOLLOWER]: Leader election won for term 1
W20260929 03:01:14.148177 22625 tablet.cc:2369] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:01:14.148602 22684 raft_consensus.cc:686] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [term 1 LEADER]: Becoming Leader. State: Replica: c59e5dd4f4514345b409abb7f51e8cfe, State: Running, Role: LEADER
I20260929 03:01:14.149204 22684 consensus_queue.cc:237] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } }
I20260929 03:01:14.152910 22702 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader c59e5dd4f4514345b409abb7f51e8cfe. Latest consensus state: current_term: 1 leader_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } } }
I20260929 03:01:14.153302 22702 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:14.156817 22597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } attempt: 5
I20260929 03:01:14.157485 22705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:01:14.159773 22705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 03:01:14 dist-test-slave-m1rv krb5kdc[22443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650873, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.125@KRBTEST.COM for kudu/127.8.208.124@KRBTEST.COM
I20260929 03:01:14.177664 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:14.157496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:37589 (local address 127.0.0.1:51848)
0929 03:01:14.157858 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:14.157880 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:14.157995 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:14.159489 (+ 1494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:14.159495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:14.160068 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:14.160282 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:14.160296 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.161800 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.161805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:14.163290 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:14.163307 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.164432 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.164438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:14.164687 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:14.165418 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:14.165443 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:14.168316 (+ 2873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:14.174019 (+ 5703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:14.174025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:14.174029 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:14.174561 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:14.175290 (+ 729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:14.175294 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:14.175297 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:14.175458 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:14.176584 (+ 1126us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:14.176589 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:14.176763 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 03:01:14.176906 (+ 143us) client_negotiation.cc:240] Negotiation successful
0929 03:01:14.177267 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260929 03:01:14.179605 22498 raft_consensus.cc:1261] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [term 1 FOLLOWER]: Refusing update from remote peer c59e5dd4f4514345b409abb7f51e8cfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:01:14.181607 22702 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:01:14.182920 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:14.158294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.8.208.124:37589)
0929 03:01:14.158706 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:14.158714 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:14.158738 (+ 24us) server_negotiation.cc:407] Connection header received
0929 03:01:14.158797 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:14.158802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:14.158891 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:14.158987 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:14.160457 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.161627 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.163544 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:14.164267 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:14.164401 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:14.168537 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:14.168545 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:14.168550 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:14.168610 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:14.173793 (+ 5183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:14.174769 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:14.174802 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:14.174807 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:14.175090 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:14.175655 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:14.175687 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:14.175692 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:14.176345 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:14.181299 (+ 4954us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:14.182260 (+ 961us) server_negotiation.cc:299] Negotiation successful
0929 03:01:14.182546 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270}
W20260929 03:01:14.188035 22477 tablet.cc:2369] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:01:14.191972 22597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed: Bootstrap starting.
I20260929 03:01:14.197129 22597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:14.201738 22597 log.cc:824] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:14.205536 22698 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c59e5dd4f4514345b409abb7f51e8cfe. Latest consensus state: current_term: 1 leader_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } } }
I20260929 03:01:14.205852 22698 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:01:14.207158 22709 mvcc.cc:205] Tried to move back new op lower bound from 7334505980618862592 to 7334505980520005632. Current Snapshot: MvccSnapshot[applied={T|T < 7334505980618862592}]
I20260929 03:01:14.207635 22702 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } } }
I20260929 03:01:14.207937 22702 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:14.214962 22597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed: No bootstrap required, opened a new log
I20260929 03:01:14.216766 22705 catalog_manager.cc:1347] Generated new cluster ID: 0fb8a908458141d8acea39c1fe68e3b9
I20260929 03:01:14.216863 22705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:01:14.232851 22698 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } } }
I20260929 03:01:14.233194 22698 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:01:14.238842 22597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } }
I20260929 03:01:14.239290 22597 raft_consensus.cc:374] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:14.239416 22597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 612830e1615e467cb6c8e1256e63e4ed, State: Initialized, Role: FOLLOWER
I20260929 03:01:14.242404 22705 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:01:14.244498 22705 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:01:14.245652 22597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } }
I20260929 03:01:14.247104 22720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } } }
I20260929 03:01:14.247925 22720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:01:14.248234 22597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:01:14.248571 22570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:14.252175 22570 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [term 1 FOLLOWER]: Refusing update from remote peer c59e5dd4f4514345b409abb7f51e8cfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:01:14.252769 22571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "612830e1615e467cb6c8e1256e63e4ed"
I20260929 03:01:14.253021 22715 catalog_manager.cc:1259] Loaded cluster ID: 0fb8a908458141d8acea39c1fe68e3b9
I20260929 03:01:14.253123 22715 catalog_manager.cc:1552] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302: loading cluster ID for follower catalog manager: success
I20260929 03:01:14.253489 22572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "612830e1615e467cb6c8e1256e63e4ed" is_pre_election: true
I20260929 03:01:14.254735 22702 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:01:14.256361 22571 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c59e5dd4f4514345b409abb7f51e8cfe in term 1.
I20260929 03:01:14.262428 22705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe: Generated new TSK 0
I20260929 03:01:14.263831 22705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:01:14.235915 22456 master_runner.cc:186] Error getting master registration for 127.8.208.125:35647: OK, instance_id { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" instance_seqno: 1790650873427982 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 03:01:14.272047 22456 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 03:01:14.271749 22715 catalog_manager.cc:1574] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302: acquiring CA information for follower catalog manager: success
I20260929 03:01:14.275384 22715 catalog_manager.cc:1602] T 00000000000000000000000000000000 P df7d3458732d477995671cdbb9104302: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 03:01:14.310721 22720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader c59e5dd4f4514345b409abb7f51e8cfe. Latest consensus state: current_term: 1 leader_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } } }
I20260929 03:01:14.311072 22720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:01:14.311398 22727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } } }
I20260929 03:01:14.311833 22727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:01:14.313124 22684 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7d3458732d477995671cdbb9104302" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45723 } } peers { permanent_uuid: "612830e1615e467cb6c8e1256e63e4ed" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35647 } } peers { permanent_uuid: "c59e5dd4f4514345b409abb7f51e8cfe" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 37589 } } }
I20260929 03:01:14.313542 22684 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59e5dd4f4514345b409abb7f51e8cfe [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:14.366645 22739 catalog_manager.cc:1259] Loaded cluster ID: 0fb8a908458141d8acea39c1fe68e3b9
I20260929 03:01:14.366824 22739 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed: loading cluster ID for follower catalog manager: success
I20260929 03:01:14.370631 22739 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed: acquiring CA information for follower catalog manager: success
I20260929 03:01:14.373203 22739 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 612830e1615e467cb6c8e1256e63e4ed: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 03:01:14.391582 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 22456
I20260929 03:01:14.392822 22518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:14.556942 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 22456
I20260929 03:01:14.583283 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 22530
I20260929 03:01:14.584746 22592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:14.699606 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 22530
I20260929 03:01:14.717794 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 22604
I20260929 03:01:14.718842 22666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:14.827319 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 22604
2026-09-29T03:01:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3635 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:01:14 dist-test-slave-m1rv krb5kdc[22746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:01:14 dist-test-slave-m1rv krb5kdc[22746](info): set up 2 sockets
Sep 29 03:01:14 dist-test-slave-m1rv krb5kdc[22746](info): commencing operation
krb5kdc: starting...
W20260929 03:01:15.925654 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.007s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:01:16 dist-test-slave-m1rv krb5kdc[22746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:01:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:01:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:01:16.148720 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:45945
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:45945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:16.476676 22759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:16.477030 22759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:16.477240 22759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:16.484584 22759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:01:16.484769 22759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:16.484905 22759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:16.485016 22759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:01:16.485105 22759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:01:16.495052 22759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:45945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:45945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:16.496438 22759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:16.497884 22759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:16.509831 22767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:16.509930 22764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:16.509860 22765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:16.510980 22759 server_base.cc:1034] running on GCE node
I20260929 03:01:16.511723 22759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:16.513103 22759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:16.514343 22759 hybrid_clock.cc:648] HybridClock initialized: now 1790650876514293 us; error 43 us; skew 500 ppm
I20260929 03:01:16.514694 22759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:16 dist-test-slave-m1rv krb5kdc[22746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650876, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:16.517727 22759 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:01:16.519151 22759 webserver.cc:460] Webserver started at http://127.8.208.126:32951/ using document root <none> and password file <none>
I20260929 03:01:16.519762 22759 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:16.519865 22759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:16.520198 22759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:01:16.523132 22759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "489b2d980d2f4d81a664a8c5902ac9ee"
format_stamp: "Formatted at 2026-09-29 03:01:16 on dist-test-slave-m1rv"
server_key: "38e5fafb19d6f399c96451efb0391161"
server_key_iv: "829945c0f210d901355b6d019b5e83f5"
server_key_version: "encryptionkey@0"
I20260929 03:01:16.523815 22759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "489b2d980d2f4d81a664a8c5902ac9ee"
format_stamp: "Formatted at 2026-09-29 03:01:16 on dist-test-slave-m1rv"
server_key: "38e5fafb19d6f399c96451efb0391161"
server_key_iv: "829945c0f210d901355b6d019b5e83f5"
server_key_version: "encryptionkey@0"
I20260929 03:01:16.528719 22759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 03:01:16.532087 22774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:16.533483 22759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:01:16.533613 22759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "489b2d980d2f4d81a664a8c5902ac9ee"
format_stamp: "Formatted at 2026-09-29 03:01:16 on dist-test-slave-m1rv"
server_key: "38e5fafb19d6f399c96451efb0391161"
server_key_iv: "829945c0f210d901355b6d019b5e83f5"
server_key_version: "encryptionkey@0"
I20260929 03:01:16.533772 22759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:16.561436 22759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:16.565377 22759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:16.565608 22759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:16.584092 22759 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:45945
I20260929 03:01:16.584117 22825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:45945 every 8 connection(s)
I20260929 03:01:16.585606 22759 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:01:16.589334 22826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:16.589462 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 22759
I20260929 03:01:16.589702 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:01:16.590356 9025 external_mini_cluster.cc:1428] Setting key 12cfd0d133fcd9b3e34e7bc59a133b4b
I20260929 03:01:16.603253 22826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee: Bootstrap starting.
Sep 29 03:01:16 dist-test-slave-m1rv krb5kdc[22746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:16.606490 22826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:16.607573 22826 log.cc:824] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:16.610354 22826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee: No bootstrap required, opened a new log
I20260929 03:01:16.613157 22829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:16.592162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56220 (local address 127.8.208.126:45945)
0929 03:01:16.592960 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:16.592975 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:16.593038 (+ 63us) server_negotiation.cc:407] Connection header received
0929 03:01:16.594841 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:16.594878 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:16.595366 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:16.595812 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:16.597229 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:16.599078 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:16.600389 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:16.601491 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:16.601676 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:16.605710 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:16.605720 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:16.605736 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:16.605778 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:16.609103 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:16.609860 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:16.609869 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:16.609876 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:16.610048 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:16.610436 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:16.610441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:16.610444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:16.610956 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:16.611296 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:16.611737 (+ 441us) server_negotiation.cc:299] Negotiation successful
0929 03:01:16.612175 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":299,"threads_started":1}
I20260929 03:01:16.617164 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:01:16.618939 22826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489b2d980d2f4d81a664a8c5902ac9ee" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45945 } }
I20260929 03:01:16.619366 22826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:16.619480 22826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 489b2d980d2f4d81a664a8c5902ac9ee, State: Initialized, Role: FOLLOWER
I20260929 03:01:16.620105 22826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489b2d980d2f4d81a664a8c5902ac9ee" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45945 } }
I20260929 03:01:16.620385 22826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:01:16.620537 22826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:01:16.620708 22826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:16.622825 22826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489b2d980d2f4d81a664a8c5902ac9ee" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45945 } }
I20260929 03:01:16.623294 22826 leader_election.cc:302] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 489b2d980d2f4d81a664a8c5902ac9ee; no voters:
I20260929 03:01:16.623858 22826 leader_election.cc:288] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:01:16.623980 22831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:01:16.624723 22831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [term 1 LEADER]: Becoming Leader. State: Replica: 489b2d980d2f4d81a664a8c5902ac9ee, State: Running, Role: LEADER
I20260929 03:01:16.625322 22831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489b2d980d2f4d81a664a8c5902ac9ee" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45945 } }
I20260929 03:01:16.625929 22826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:01:16.628106 22832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "489b2d980d2f4d81a664a8c5902ac9ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489b2d980d2f4d81a664a8c5902ac9ee" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45945 } } }
I20260929 03:01:16.628580 22832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:16.628157 22833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 489b2d980d2f4d81a664a8c5902ac9ee. Latest consensus state: current_term: 1 leader_uuid: "489b2d980d2f4d81a664a8c5902ac9ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489b2d980d2f4d81a664a8c5902ac9ee" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 45945 } } }
I20260929 03:01:16.628702 22833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:16.630309 22839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:01:16.635489 22839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:01:16.644057 22839 catalog_manager.cc:1347] Generated new cluster ID: de055e14e6404a95b73478bd0241c6b7
I20260929 03:01:16.644173 22839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:01:16.660307 22839 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:01:16.662082 22839 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:01:16.676985 22839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 489b2d980d2f4d81a664a8c5902ac9ee: Generated new TSK 0
I20260929 03:01:16.678081 22839 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:01:16.729568 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 22759
I20260929 03:01:16.730799 22821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:16.827983 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 22759
2026-09-29T03:01:16Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1994 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:01:16 dist-test-slave-m1rv krb5kdc[22854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:01:16 dist-test-slave-m1rv krb5kdc[22854](info): set up 2 sockets
Sep 29 03:01:16 dist-test-slave-m1rv krb5kdc[22854](info): commencing operation
krb5kdc: starting...
W20260929 03:01:17.925675 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.007s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:01:18 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:01:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:01:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:01:18.146492 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:46583
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:46583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:18.471962 22867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:18.472278 22867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:18.472458 22867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:18.479637 22867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:01:18.479790 22867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:18.479902 22867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:18.480002 22867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:01:18.480083 22867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:01:18.489760 22867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:46583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:46583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:18.491130 22867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:18.492458 22867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:18.503978 22872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:18.504074 22873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:18.504123 22875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:18.505062 22867 server_base.cc:1034] running on GCE node
I20260929 03:01:18.505841 22867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:18.507230 22867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:18.508442 22867 hybrid_clock.cc:648] HybridClock initialized: now 1790650878508390 us; error 39 us; skew 500 ppm
I20260929 03:01:18.508786 22867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:18 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650878, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:18.516630 22867 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:01:18.518280 22867 webserver.cc:460] Webserver started at http://127.8.208.126:46435/ using document root <none> and password file <none>
I20260929 03:01:18.518949 22867 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:18.519088 22867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:18.519373 22867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:01:18.522336 22867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "015013a3cf5e49439b418017f6eadfe0"
format_stamp: "Formatted at 2026-09-29 03:01:18 on dist-test-slave-m1rv"
server_key: "f62b34461289cd22dbc1506282218e8a"
server_key_iv: "f4a9908fb41f386bd5b048924296b523"
server_key_version: "encryptionkey@0"
I20260929 03:01:18.523087 22867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "015013a3cf5e49439b418017f6eadfe0"
format_stamp: "Formatted at 2026-09-29 03:01:18 on dist-test-slave-m1rv"
server_key: "f62b34461289cd22dbc1506282218e8a"
server_key_iv: "f4a9908fb41f386bd5b048924296b523"
server_key_version: "encryptionkey@0"
I20260929 03:01:18.528077 22867 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 03:01:18.531383 22882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:18.532545 22867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260929 03:01:18.532732 22867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "015013a3cf5e49439b418017f6eadfe0"
format_stamp: "Formatted at 2026-09-29 03:01:18 on dist-test-slave-m1rv"
server_key: "f62b34461289cd22dbc1506282218e8a"
server_key_iv: "f4a9908fb41f386bd5b048924296b523"
server_key_version: "encryptionkey@0"
I20260929 03:01:18.532919 22867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:18.556499 22867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:18.560894 22867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:18.561215 22867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:18.579237 22867 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:46583
I20260929 03:01:18.579279 22933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:46583 every 8 connection(s)
I20260929 03:01:18.580562 22867 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:01:18.584004 22934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:18.587671 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 22867
I20260929 03:01:18.587893 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:01:18.588825 9025 external_mini_cluster.cc:1428] Setting key dc011e6c38a3e708f1eb7a48a80ba4a0
I20260929 03:01:18.597434 22934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Bootstrap starting.
I20260929 03:01:18.601118 22934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:18.602495 22934 log.cc:824] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Log is configured to *not* fsync() on all Append() calls
Sep 29 03:01:18 dist-test-slave-m1rv krb5kdc[22854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:18.605439 22934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: No bootstrap required, opened a new log
I20260929 03:01:18.611997 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:18.590930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.8.208.126:46583)
0929 03:01:18.591668 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:18.591680 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:18.591717 (+ 37us) server_negotiation.cc:407] Connection header received
0929 03:01:18.593367 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:18.593389 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:18.593773 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:18.594082 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:18.595815 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:18.597061 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:18.598913 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:18.599585 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:18.599717 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:18.604738 (+ 5021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:18.604746 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:18.604761 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:18.604799 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:18.607942 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:18.608678 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:18.608695 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:18.608700 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:18.608856 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:18.609328 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:18.609332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:18.609334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:18.609853 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:18.610203 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:18.610592 (+ 389us) server_negotiation.cc:299] Negotiation successful
0929 03:01:18.611028 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":345,"threads_started":1}
I20260929 03:01:18.613240 22934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } }
I20260929 03:01:18.613765 22934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:18.613883 22934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015013a3cf5e49439b418017f6eadfe0, State: Initialized, Role: FOLLOWER
I20260929 03:01:18.614506 22934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } }
I20260929 03:01:18.614790 22934 raft_consensus.cc:388] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:01:18.614941 22934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:01:18.615111 22934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:18.617286 22934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } }
I20260929 03:01:18.617743 22934 leader_election.cc:302] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 015013a3cf5e49439b418017f6eadfe0; no voters:
I20260929 03:01:18.618405 22934 leader_election.cc:288] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:01:18.618469 22939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:01:18.619779 22939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 1 LEADER]: Becoming Leader. State: Replica: 015013a3cf5e49439b418017f6eadfe0, State: Running, Role: LEADER
I20260929 03:01:18.620278 22939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } }
I20260929 03:01:18.620494 22934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:01:18.623339 22941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 015013a3cf5e49439b418017f6eadfe0. Latest consensus state: current_term: 1 leader_uuid: "015013a3cf5e49439b418017f6eadfe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } } }
I20260929 03:01:18.623785 22941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:18.624446 22940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "015013a3cf5e49439b418017f6eadfe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } } }
I20260929 03:01:18.624854 22940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:18.627511 22948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:01:18.632416 22948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:01:18.640349 22948 catalog_manager.cc:1347] Generated new cluster ID: fd861c2da81844a7964692655b07fecc
I20260929 03:01:18.640460 22948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:01:18.657575 22948 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:01:18.658811 22948 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:01:18.669268 22948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Generated new TSK 0
I20260929 03:01:18.670073 22948 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 03:01:18.732681 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46583
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:19.062649 22960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:19.062947 22960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:19.063050 22960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:19.071316 22960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:19.071448 22960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:19.071574 22960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:01:19.082602 22960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:19.083915 22960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:19.085278 22960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:19.099970 22968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:19.099972 22966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:19.100229 22965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:19.101444 22960 server_base.cc:1034] running on GCE node
I20260929 03:01:19.102064 22960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:19.103003 22960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:19.104234 22960 hybrid_clock.cc:648] HybridClock initialized: now 1790650879104181 us; error 38 us; skew 500 ppm
I20260929 03:01:19.104578 22960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:19 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650879, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:19.107836 22960 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 03:01:19.109622 22960 webserver.cc:460] Webserver started at http://127.8.208.65:35591/ using document root <none> and password file <none>
I20260929 03:01:19.110349 22960 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:19.110458 22960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:19.110708 22960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:01:19.113898 22960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "a96430697e744447ad89aea3e4ff7a3e"
format_stamp: "Formatted at 2026-09-29 03:01:19 on dist-test-slave-m1rv"
server_key: "ad469d81ee639024837d822109b905a4"
server_key_iv: "b08e1a2da0d69de285a6745c3717b30c"
server_key_version: "encryptionkey@0"
I20260929 03:01:19.114593 22960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "a96430697e744447ad89aea3e4ff7a3e"
format_stamp: "Formatted at 2026-09-29 03:01:19 on dist-test-slave-m1rv"
server_key: "ad469d81ee639024837d822109b905a4"
server_key_iv: "b08e1a2da0d69de285a6745c3717b30c"
server_key_version: "encryptionkey@0"
I20260929 03:01:19.119788 22960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 03:01:19.123307 22975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:19.124804 22960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:01:19.124964 22960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "a96430697e744447ad89aea3e4ff7a3e"
format_stamp: "Formatted at 2026-09-29 03:01:19 on dist-test-slave-m1rv"
server_key: "ad469d81ee639024837d822109b905a4"
server_key_iv: "b08e1a2da0d69de285a6745c3717b30c"
server_key_version: "encryptionkey@0"
I20260929 03:01:19.125165 22960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:19.138163 22960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:19.142606 22960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:19.142897 22960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:19.143813 22960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:19.145113 22960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:19.145242 22960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260929 03:01:19.145380 22960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:19.145459 22960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:19.178107 22960 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:35459
I20260929 03:01:19.178140 23087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:35459 every 8 connection(s)
I20260929 03:01:19.179462 22960 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:01:19.188400 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 22960
I20260929 03:01:19.188611 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 03:01:19.189343 9025 external_mini_cluster.cc:1428] Setting key 876cb7abc449ba0ea957a80b23932f8e
Sep 29 03:01:19 dist-test-slave-m1rv krb5kdc[22854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650879, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:19.200775 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:19.182052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:58947 (local address 127.8.208.126:46583)
0929 03:01:19.182494 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:19.182500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:19.183485 (+ 985us) server_negotiation.cc:407] Connection header received
0929 03:01:19.185328 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:19.185332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:19.185402 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:19.185504 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:19.186984 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:19.187674 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:19.189407 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:19.190006 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:19.190113 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:19.193996 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:19.194003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:19.194009 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:19.194056 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:19.197711 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:19.198400 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:19.198408 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:19.198415 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:19.198574 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:19.198971 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:19.198980 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:19.198986 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:19.199318 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:19.199451 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:19.200350 (+ 899us) server_negotiation.cc:299] Negotiation successful
0929 03:01:19.200524 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":207,"threads_started":1}
I20260929 03:01:19.202335 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:19.182624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:58947)
0929 03:01:19.183327 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:19.183365 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:19.184994 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:19.185722 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:19.185731 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:19.186261 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:19.186835 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:19.186852 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:19.187809 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:19.187812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:19.189256 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:19.189269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:19.190246 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:19.190253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:19.190488 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:19.191249 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:19.191283 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:19.193774 (+ 2491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:19.197843 (+ 4069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:19.197850 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:19.197863 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:19.198275 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:19.198676 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:19.198679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:19.198682 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:19.198856 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:19.199463 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:19.199472 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:19.199927 (+ 455us) client_negotiation.cc:769] Sending connection context
0929 03:01:19.200268 (+ 341us) client_negotiation.cc:240] Negotiation successful
0929 03:01:19.200847 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":272,"threads_started":1}
I20260929 03:01:19.203850 23088 heartbeater.cc:347] Connected to a master server at 127.8.208.126:46583
I20260929 03:01:19.204154 23088 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:19.204877 23088 heartbeater.cc:511] Master 127.8.208.126:46583 requested a full tablet report, sending...
I20260929 03:01:19.206933 22899 ts_manager.cc:194] Registered new tserver with Master: a96430697e744447ad89aea3e4ff7a3e (127.8.208.65:35459)
I20260929 03:01:19.208391 22899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:58947
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 03:01:19.244673 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46583
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:19.575853 23094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:19.576159 23094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:19.576256 23094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:19.583801 23094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:19.583920 23094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:19.584143 23094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:01:19.594267 23094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:19.595608 23094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:19.596962 23094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:19.610455 23099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:19.610530 23102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:19.610425 23100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:19.611157 23094 server_base.cc:1034] running on GCE node
I20260929 03:01:19.611768 23094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:19.612712 23094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:19.613921 23094 hybrid_clock.cc:648] HybridClock initialized: now 1790650879613870 us; error 49 us; skew 500 ppm
I20260929 03:01:19.614219 23094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:19 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650879, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:19.617239 23094 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 03:01:19.618548 23094 webserver.cc:460] Webserver started at http://127.8.208.66:46839/ using document root <none> and password file <none>
I20260929 03:01:19.619177 23094 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:19.619269 23094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:19.619515 23094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:01:19.622752 23094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "388947b5ad3b4a72b7a37b33b2104bc2"
format_stamp: "Formatted at 2026-09-29 03:01:19 on dist-test-slave-m1rv"
server_key: "f2d7c80c77a02aad05bdd4e53c3df9c8"
server_key_iv: "a19986706b7007669601687066799984"
server_key_version: "encryptionkey@0"
I20260929 03:01:19.623428 23094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "388947b5ad3b4a72b7a37b33b2104bc2"
format_stamp: "Formatted at 2026-09-29 03:01:19 on dist-test-slave-m1rv"
server_key: "f2d7c80c77a02aad05bdd4e53c3df9c8"
server_key_iv: "a19986706b7007669601687066799984"
server_key_version: "encryptionkey@0"
I20260929 03:01:19.628515 23094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 03:01:19.631847 23109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:19.633107 23094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 03:01:19.633324 23094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "388947b5ad3b4a72b7a37b33b2104bc2"
format_stamp: "Formatted at 2026-09-29 03:01:19 on dist-test-slave-m1rv"
server_key: "f2d7c80c77a02aad05bdd4e53c3df9c8"
server_key_iv: "a19986706b7007669601687066799984"
server_key_version: "encryptionkey@0"
I20260929 03:01:19.633488 23094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:19.652249 23094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:19.655977 23094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:19.656280 23094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:19.657212 23094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:19.658520 23094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:19.658591 23094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:19.658752 23094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:19.658803 23094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:19.691063 23094 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:43993
I20260929 03:01:19.691100 23221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:43993 every 8 connection(s)
I20260929 03:01:19.692297 23094 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:01:19.696717 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 23094
I20260929 03:01:19.696907 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 03:01:19.697389 9025 external_mini_cluster.cc:1428] Setting key d8fde2265d8a00872f97fecf1617d3e2
Sep 29 03:01:19 dist-test-slave-m1rv krb5kdc[22854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650879, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:19.713086 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:19.694762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:49637 (local address 127.8.208.126:46583)
0929 03:01:19.694941 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:19.694945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:19.696073 (+ 1128us) server_negotiation.cc:407] Connection header received
0929 03:01:19.698025 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:19.698033 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:19.698130 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:19.698219 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:19.699783 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:19.701022 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:19.702602 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:19.703417 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:19.703576 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:19.707178 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:19.707183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:19.707187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:19.707221 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:19.709878 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:19.710805 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:19.710809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:19.710811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:19.710936 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:19.711408 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:19.711411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:19.711413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:19.711657 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:19.711761 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:19.712663 (+ 902us) server_negotiation.cc:299] Negotiation successful
0929 03:01:19.712841 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 03:01:19.714712 23224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:19.695262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:49637)
0929 03:01:19.695919 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:19.695954 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:19.697753 (+ 1799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:19.698476 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:19.698484 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:19.699046 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:19.699614 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:19.699631 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:19.701231 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:19.701235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:19.702458 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:19.702470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:19.703626 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:19.703630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:19.703830 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:19.704537 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:19.704564 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:19.706943 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:19.710013 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:19.710025 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:19.710054 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:19.710684 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:19.711046 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:19.711054 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:19.711060 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:19.711303 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:19.711776 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:19.711793 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:19.712263 (+ 470us) client_negotiation.cc:769] Sending connection context
0929 03:01:19.712592 (+ 329us) client_negotiation.cc:240] Negotiation successful
0929 03:01:19.713233 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":263,"threads_started":1}
I20260929 03:01:19.716135 23222 heartbeater.cc:347] Connected to a master server at 127.8.208.126:46583
I20260929 03:01:19.716465 23222 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:19.717592 23222 heartbeater.cc:511] Master 127.8.208.126:46583 requested a full tablet report, sending...
I20260929 03:01:19.719256 22899 ts_manager.cc:194] Registered new tserver with Master: 388947b5ad3b4a72b7a37b33b2104bc2 (127.8.208.66:43993)
I20260929 03:01:19.720198 22899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:49637
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 03:01:19.753116 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46583
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:20.074883 23227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:20.075204 23227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:20.075369 23227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:20.082546 23227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:20.082688 23227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:20.082861 23227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:01:20.092512 23227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:20.093879 23227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:20.095218 23227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:20.108294 23232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:20.108258 23235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:20.108217 23233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:20.109216 23227 server_base.cc:1034] running on GCE node
I20260929 03:01:20.109771 23227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:20.110689 23227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:20.111901 23227 hybrid_clock.cc:648] HybridClock initialized: now 1790650880111863 us; error 32 us; skew 500 ppm
I20260929 03:01:20.112203 23227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:20 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650880, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:20.115166 23227 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 03:01:20.116835 23227 webserver.cc:460] Webserver started at http://127.8.208.67:41381/ using document root <none> and password file <none>
I20260929 03:01:20.117558 23227 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:20.117688 23227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:20.118002 23227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:01:20.120868 23227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "b2a0147c9fd749678504d5f360795338"
format_stamp: "Formatted at 2026-09-29 03:01:20 on dist-test-slave-m1rv"
server_key: "c22dc6fb70fd6aeaf1d90b28989b002d"
server_key_iv: "70fa2a7177bbb470d253514559d07887"
server_key_version: "encryptionkey@0"
I20260929 03:01:20.121758 23227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "b2a0147c9fd749678504d5f360795338"
format_stamp: "Formatted at 2026-09-29 03:01:20 on dist-test-slave-m1rv"
server_key: "c22dc6fb70fd6aeaf1d90b28989b002d"
server_key_iv: "70fa2a7177bbb470d253514559d07887"
server_key_version: "encryptionkey@0"
I20260929 03:01:20.126729 23227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 03:01:20.130213 23242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:20.131493 23227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:01:20.131661 23227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "b2a0147c9fd749678504d5f360795338"
format_stamp: "Formatted at 2026-09-29 03:01:20 on dist-test-slave-m1rv"
server_key: "c22dc6fb70fd6aeaf1d90b28989b002d"
server_key_iv: "70fa2a7177bbb470d253514559d07887"
server_key_version: "encryptionkey@0"
I20260929 03:01:20.131853 23227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:20.154300 23227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:20.158123 23227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:20.158411 23227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:20.159324 23227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:20.160612 23227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:20.160717 23227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:20.160862 23227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:20.160933 23227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:20.193951 23227 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:35347
I20260929 03:01:20.194038 23354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:35347 every 8 connection(s)
I20260929 03:01:20.195240 23227 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:01:20.203735 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 23227
I20260929 03:01:20.203876 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 03:01:20.204312 9025 external_mini_cluster.cc:1428] Setting key e807ecd15ad740c0dbf32102b2b12a07
Sep 29 03:01:20 dist-test-slave-m1rv krb5kdc[22854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650880, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:20.211236 23088 heartbeater.cc:503] Master 127.8.208.126:46583 was elected leader, sending a full tablet report...
I20260929 03:01:20.215494 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.197775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:55525 (local address 127.8.208.126:46583)
0929 03:01:20.197954 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:20.197959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:20.199060 (+ 1101us) server_negotiation.cc:407] Connection header received
0929 03:01:20.200870 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:20.200876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:20.200987 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:20.201109 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:20.202577 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.203331 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.205293 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.205868 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.206017 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.209696 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:20.209700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:20.209704 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:20.209736 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:20.212548 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:20.213286 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:20.213295 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:20.213297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:20.213420 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:20.213850 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:20.213855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:20.213858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:20.214118 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:20.214273 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:20.214938 (+ 665us) server_negotiation.cc:299] Negotiation successful
0929 03:01:20.215159 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:01:20.216423 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.198162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:55525)
0929 03:01:20.198898 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:20.198939 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:20.200607 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:20.201355 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:20.201363 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:20.201856 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:20.202404 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:20.202424 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.203505 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.203510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:20.205129 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:20.205142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.206086 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.206094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:20.206405 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.207143 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:20.207174 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:20.209506 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:20.212681 (+ 3175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:20.212690 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:20.212706 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:20.213120 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:20.213540 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:20.213545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:20.213548 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:20.213736 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:20.214247 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:20.214255 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:20.214582 (+ 327us) client_negotiation.cc:769] Sending connection context
0929 03:01:20.214832 (+ 250us) client_negotiation.cc:240] Negotiation successful
0929 03:01:20.215337 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":290,"threads_started":1}
I20260929 03:01:20.217793 23355 heartbeater.cc:347] Connected to a master server at 127.8.208.126:46583
I20260929 03:01:20.218114 23355 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:20.218875 23355 heartbeater.cc:511] Master 127.8.208.126:46583 requested a full tablet report, sending...
I20260929 03:01:20.220187 22899 ts_manager.cc:194] Registered new tserver with Master: b2a0147c9fd749678504d5f360795338 (127.8.208.67:35347)
I20260929 03:01:20.221108 22899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:55525
I20260929 03:01:20.231530 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:01:20.248040 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.235935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36818 (local address 127.8.208.126:46583)
0929 03:01:20.236143 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:20.236152 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:20.236577 (+ 425us) server_negotiation.cc:407] Connection header received
0929 03:01:20.236710 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:20.236714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:20.236773 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:20.236846 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:20.238362 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.239075 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.240342 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.240732 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.240824 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.243139 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:20.243144 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:20.243147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:20.243180 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:20.245815 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:20.246391 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:20.246395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:20.246397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:20.246520 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:20.246840 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:20.246843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:20.246845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:20.247136 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:20.247316 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:20.247547 (+ 231us) server_negotiation.cc:299] Negotiation successful
0929 03:01:20.247703 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:01:20.251775 22899 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:01:20.258577 22899 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:01:20.291695 23370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.270891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35459 (local address 127.0.0.1:34270)
0929 03:01:20.282313 (+ 11422us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:20.282332 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:20.282469 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:20.283314 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:20.283321 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:20.283360 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:20.283762 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:20.283776 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.286039 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.286046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:20.288537 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:20.288549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.290645 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.290649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:20.290779 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.290813 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 03:01:20.290971 (+ 158us) client_negotiation.cc:240] Negotiation successful
0929 03:01:20.291165 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11262,"thread_start_us":11019,"threads_started":1}
I20260929 03:01:20.291690 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.271357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.8.208.65:35459)
0929 03:01:20.281206 (+ 9849us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:20.281217 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:20.282600 (+ 1383us) server_negotiation.cc:407] Connection header received
0929 03:01:20.282710 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:20.282718 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:20.283073 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:20.283314 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:20.283989 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.285888 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.288712 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.290226 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.290363 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.290542 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:20.291008 (+ 466us) server_negotiation.cc:299] Negotiation successful
0929 03:01:20.291174 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9616,"thread_start_us":10098,"threads_started":1}
I20260929 03:01:20.292706 23369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.270539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35347 (local address 127.0.0.1:34762)
0929 03:01:20.281127 (+ 10588us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:20.281152 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:20.281348 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:20.284675 (+ 3327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:20.284680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:20.284707 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:20.285215 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:20.285274 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.287420 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.287427 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:20.289808 (+ 2381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:20.289824 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.291796 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.291861 (+ 65us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:20.292063 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.292102 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 03:01:20.292165 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 03:01:20.292301 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10440,"mutex_wait_us":47,"thread_start_us":10306,"threads_started":1}
I20260929 03:01:20.292933 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.269749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:43993 (local address 127.0.0.1:48760)
0929 03:01:20.270463 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:20.270500 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:20.270682 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:20.285006 (+ 14324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:20.285010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:20.285040 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:20.285628 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:20.285641 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.287918 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.287924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:20.290157 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:20.290170 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.292093 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.292098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:20.292261 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.292288 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 03:01:20.292371 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 03:01:20.292505 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":4,"thread_start_us":331,"threads_started":1}
I20260929 03:01:20.293956 23368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.269945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48760 (local address 127.8.208.66:43993)
0929 03:01:20.284411 (+ 14466us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:20.284417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:20.284440 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:01:20.284512 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:20.284517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:20.284716 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:20.284998 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:20.285940 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.287743 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.290390 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.291870 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.293264 (+ 1394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.293449 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:20.293508 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 03:01:20.293658 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14315,"thread_start_us":14207,"threads_started":1}
I20260929 03:01:20.293977 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.271326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.8.208.67:35347)
0929 03:01:20.284076 (+ 12750us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:20.284088 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:20.284121 (+ 33us) server_negotiation.cc:407] Connection header received
0929 03:01:20.284219 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:20.284228 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:20.284459 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:20.284668 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:20.285713 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.287264 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.289999 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.291546 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.293118 (+ 1572us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.293384 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:20.293456 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 03:01:20.293627 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12572,"thread_start_us":12391,"threads_started":1}
I20260929 03:01:20.296208 23023 tablet_service.cc:1467] Processing CreateTablet for tablet 9b17f27457aa4cfeaa2b1d1a349164a5 (DEFAULT_TABLE table=test-table [id=093f97d0487a494ba1a423c0af169556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:01:20.297729 23023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b17f27457aa4cfeaa2b1d1a349164a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:20.297755 23290 tablet_service.cc:1467] Processing CreateTablet for tablet 9b17f27457aa4cfeaa2b1d1a349164a5 (DEFAULT_TABLE table=test-table [id=093f97d0487a494ba1a423c0af169556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:01:20.297765 23157 tablet_service.cc:1467] Processing CreateTablet for tablet 9b17f27457aa4cfeaa2b1d1a349164a5 (DEFAULT_TABLE table=test-table [id=093f97d0487a494ba1a423c0af169556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:01:20.299357 23290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b17f27457aa4cfeaa2b1d1a349164a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:20.299371 23157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b17f27457aa4cfeaa2b1d1a349164a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:20.310786 23373 tablet_bootstrap.cc:492] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e: Bootstrap starting.
I20260929 03:01:20.313925 23373 tablet_bootstrap.cc:654] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:20.314915 23373 log.cc:824] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:20.317328 23375 tablet_bootstrap.cc:492] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338: Bootstrap starting.
I20260929 03:01:20.318449 23374 tablet_bootstrap.cc:492] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2: Bootstrap starting.
I20260929 03:01:20.319970 23373 tablet_bootstrap.cc:492] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e: No bootstrap required, opened a new log
I20260929 03:01:20.320472 23373 ts_tablet_manager.cc:1397] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260929 03:01:20.321509 23375 tablet_bootstrap.cc:654] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:20.322525 23374 tablet_bootstrap.cc:654] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:20.322975 23375 log.cc:824] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:20.323767 23374 log.cc:824] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:20.327267 23375 tablet_bootstrap.cc:492] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338: No bootstrap required, opened a new log
I20260929 03:01:20.327597 23375 ts_tablet_manager.cc:1397] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260929 03:01:20.328945 23373 raft_consensus.cc:348] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:20.329574 23373 raft_consensus.cc:374] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:20.329689 23373 raft_consensus.cc:729] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96430697e744447ad89aea3e4ff7a3e, State: Initialized, Role: FOLLOWER
I20260929 03:01:20.330350 23373 consensus_queue.cc:260] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:20.331900 23373 ts_tablet_manager.cc:1428] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260929 03:01:20.333447 23374 tablet_bootstrap.cc:492] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2: No bootstrap required, opened a new log
I20260929 03:01:20.333719 23374 ts_tablet_manager.cc:1397] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260929 03:01:20.338572 23375 raft_consensus.cc:348] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:20.339041 23375 raft_consensus.cc:374] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:20.339143 23375 raft_consensus.cc:729] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a0147c9fd749678504d5f360795338, State: Initialized, Role: FOLLOWER
I20260929 03:01:20.339749 23375 consensus_queue.cc:260] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:20.340785 23355 heartbeater.cc:503] Master 127.8.208.126:46583 was elected leader, sending a full tablet report...
I20260929 03:01:20.341530 23375 ts_tablet_manager.cc:1428] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260929 03:01:20.344259 23374 raft_consensus.cc:348] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:20.344689 23374 raft_consensus.cc:374] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:20.344792 23374 raft_consensus.cc:729] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 388947b5ad3b4a72b7a37b33b2104bc2, State: Initialized, Role: FOLLOWER
I20260929 03:01:20.345434 23374 consensus_queue.cc:260] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:20.346386 23222 heartbeater.cc:503] Master 127.8.208.126:46583 was elected leader, sending a full tablet report...
I20260929 03:01:20.346812 23374 ts_tablet_manager.cc:1428] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
W20260929 03:01:20.431802 23089 tablet.cc:2369] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:01:20.444237 23223 tablet.cc:2369] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:01:20.446806 23356 tablet.cc:2369] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:01:20.495348 23381 raft_consensus.cc:482] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:01:20.495699 23381 raft_consensus.cc:504] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:20.497392 23381 leader_election.cc:288] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a96430697e744447ad89aea3e4ff7a3e (127.8.208.65:35459), b2a0147c9fd749678504d5f360795338 (127.8.208.67:35347)
I20260929 03:01:20.504964 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.497935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:59015 (local address 127.8.208.67:35347)
0929 03:01:20.498167 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:20.498175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:20.498925 (+ 750us) server_negotiation.cc:407] Connection header received
0929 03:01:20.499061 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:20.499066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:20.499145 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:20.499229 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:20.499823 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.500608 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.503255 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.504042 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.504204 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.504279 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:20.504585 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 03:01:20.504711 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 03:01:20.504969 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.497932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:53363 (local address 127.8.208.65:35459)
0929 03:01:20.498135 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:20.498142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:20.498598 (+ 456us) server_negotiation.cc:407] Connection header received
0929 03:01:20.498841 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:20.498845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:20.498929 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:20.499025 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:20.499951 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.500640 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.503057 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.503931 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.504061 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.504147 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:20.504548 (+ 401us) server_negotiation.cc:299] Negotiation successful
0929 03:01:20.504679 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 03:01:20.505141 23382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.497728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35459 (local address 127.8.208.66:53363)
0929 03:01:20.498481 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:20.498497 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:20.498691 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:20.499357 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:20.499360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:20.499393 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:20.499803 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:20.499815 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.500799 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.500802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:20.502902 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:20.502913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.504171 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.504178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:20.504379 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.504404 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 03:01:20.504481 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 03:01:20.504623 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":7,"thread_start_us":640,"threads_started":1}
I20260929 03:01:20.505210 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.498111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35347 (local address 127.8.208.66:59015)
0929 03:01:20.498817 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:20.498832 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:20.498942 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:20.499272 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:20.499275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:20.499294 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:20.499655 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:20.499664 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.500751 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.500758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:20.503095 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:20.503108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.504275 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.504279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:20.504404 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.504426 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 03:01:20.504541 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 03:01:20.504666 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":74,"thread_start_us":435,"threads_started":1}
I20260929 03:01:20.506496 23043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b17f27457aa4cfeaa2b1d1a349164a5" candidate_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96430697e744447ad89aea3e4ff7a3e" is_pre_election: true
I20260929 03:01:20.506496 23310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b17f27457aa4cfeaa2b1d1a349164a5" candidate_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2a0147c9fd749678504d5f360795338" is_pre_election: true
I20260929 03:01:20.507077 23043 raft_consensus.cc:2454] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 388947b5ad3b4a72b7a37b33b2104bc2 in term 0.
I20260929 03:01:20.507077 23310 raft_consensus.cc:2454] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 388947b5ad3b4a72b7a37b33b2104bc2 in term 0.
I20260929 03:01:20.507758 23112 leader_election.cc:302] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388947b5ad3b4a72b7a37b33b2104bc2, b2a0147c9fd749678504d5f360795338; no voters:
I20260929 03:01:20.508176 23381 raft_consensus.cc:2781] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:01:20.508332 23381 raft_consensus.cc:482] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:01:20.508458 23381 raft_consensus.cc:3037] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:20.510751 23381 raft_consensus.cc:504] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:20.511446 23381 leader_election.cc:288] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [CANDIDATE]: Term 1 election: Requested vote from peers a96430697e744447ad89aea3e4ff7a3e (127.8.208.65:35459), b2a0147c9fd749678504d5f360795338 (127.8.208.67:35347)
I20260929 03:01:20.511876 23043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b17f27457aa4cfeaa2b1d1a349164a5" candidate_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96430697e744447ad89aea3e4ff7a3e"
I20260929 03:01:20.511979 23310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b17f27457aa4cfeaa2b1d1a349164a5" candidate_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2a0147c9fd749678504d5f360795338"
I20260929 03:01:20.512111 23043 raft_consensus.cc:3037] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:20.512179 23310 raft_consensus.cc:3037] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:20.515364 23043 raft_consensus.cc:2454] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 388947b5ad3b4a72b7a37b33b2104bc2 in term 1.
I20260929 03:01:20.515375 23310 raft_consensus.cc:2454] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 388947b5ad3b4a72b7a37b33b2104bc2 in term 1.
I20260929 03:01:20.515815 23113 leader_election.cc:302] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 388947b5ad3b4a72b7a37b33b2104bc2, a96430697e744447ad89aea3e4ff7a3e, b2a0147c9fd749678504d5f360795338; no voters:
I20260929 03:01:20.516150 23381 raft_consensus.cc:2781] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:01:20.516589 23381 raft_consensus.cc:686] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 1 LEADER]: Becoming Leader. State: Replica: 388947b5ad3b4a72b7a37b33b2104bc2, State: Running, Role: LEADER
I20260929 03:01:20.517099 23381 consensus_queue.cc:237] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:20.521546 22899 catalog_manager.cc:5495] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 388947b5ad3b4a72b7a37b33b2104bc2 (127.8.208.66). New cstate: current_term: 1 leader_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } health_report { overall_health: UNKNOWN } } }
I20260929 03:01:20.591146 23368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.585844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.8.208.66:43993)
0929 03:01:20.586063 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:20.586072 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:20.586092 (+ 20us) server_negotiation.cc:407] Connection header received
0929 03:01:20.586192 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:20.586195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:20.586268 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:20.586416 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:01:20.586796 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.587513 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.588922 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.589402 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.589572 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.590071 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:01:20.590568 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:01:20.590740 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:20.590792 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 03:01:20.590871 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 03:01:20.598953 23310 raft_consensus.cc:1261] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338 [term 1 FOLLOWER]: Refusing update from remote peer 388947b5ad3b4a72b7a37b33b2104bc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:01:20.598963 23043 raft_consensus.cc:1261] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e [term 1 FOLLOWER]: Refusing update from remote peer 388947b5ad3b4a72b7a37b33b2104bc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:01:20.599910 23384 consensus_queue.cc:1035] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:01:20.600241 23381 consensus_queue.cc:1035] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:01:20.610524 23387 mvcc.cc:205] Tried to move back new op lower bound from 7334506006923571200 to 7334506006601908224. Current Snapshot: MvccSnapshot[applied={T|T < 7334506006923571200}]
I20260929 03:01:20.610869 23388 mvcc.cc:205] Tried to move back new op lower bound from 7334506006923571200 to 7334506006601908224. Current Snapshot: MvccSnapshot[applied={T|T < 7334506006923571200}]
I20260929 03:01:20.626624 22898 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:01:20.626883 22898 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:01:20.630761 22898 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Sending DeleteTablet for 3 replicas of tablet 9b17f27457aa4cfeaa2b1d1a349164a5
I20260929 03:01:20.631949 23290 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b17f27457aa4cfeaa2b1d1a349164a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:01:20 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:34762
I20260929 03:01:20.631956 23157 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b17f27457aa4cfeaa2b1d1a349164a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:01:20 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:48760
I20260929 03:01:20.632551 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 22960
I20260929 03:01:20.632644 23023 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b17f27457aa4cfeaa2b1d1a349164a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:01:20 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:34270
I20260929 03:01:20.643102 23399 tablet_replica.cc:330] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e: stopping tablet replica
I20260929 03:01:20.644282 23399 raft_consensus.cc:2229] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:01:20.645766 23399 raft_consensus.cc:2258] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:01:20.647936 23397 tablet_replica.cc:330] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338: stopping tablet replica
I20260929 03:01:20.648531 23397 raft_consensus.cc:2229] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:01:20.648753 23399 ts_tablet_manager.cc:1905] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:01:20.649209 23397 raft_consensus.cc:2258] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:01:20.650175 23398 tablet_replica.cc:330] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2: stopping tablet replica
I20260929 03:01:20.650666 23398 raft_consensus.cc:2229] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:01:20.651429 23398 raft_consensus.cc:2258] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:01:20.654031 23398 ts_tablet_manager.cc:1905] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:01:20.655144 23397 ts_tablet_manager.cc:1905] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:01:20.655990 23399 ts_tablet_manager.cc:1918] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:01:20.656198 23399 log.cc:1196] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/9b17f27457aa4cfeaa2b1d1a349164a5
I20260929 03:01:20.656780 23399 ts_tablet_manager.cc:1939] T 9b17f27457aa4cfeaa2b1d1a349164a5 P a96430697e744447ad89aea3e4ff7a3e: Deleting consensus metadata
I20260929 03:01:20.658778 22886 catalog_manager.cc:4841] TS a96430697e744447ad89aea3e4ff7a3e (127.8.208.65:35459): tablet 9b17f27457aa4cfeaa2b1d1a349164a5 (table test-table [id=093f97d0487a494ba1a423c0af169556]) successfully deleted
I20260929 03:01:20.661576 23398 ts_tablet_manager.cc:1918] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:01:20.661803 23398 log.cc:1196] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/9b17f27457aa4cfeaa2b1d1a349164a5
I20260929 03:01:20.662410 23398 ts_tablet_manager.cc:1939] T 9b17f27457aa4cfeaa2b1d1a349164a5 P 388947b5ad3b4a72b7a37b33b2104bc2: Deleting consensus metadata
Sep 29 03:01:20 dist-test-slave-m1rv krb5kdc[22854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 03:01:20.664062 22886 catalog_manager.cc:4841] TS 388947b5ad3b4a72b7a37b33b2104bc2 (127.8.208.66:43993): tablet 9b17f27457aa4cfeaa2b1d1a349164a5 (table test-table [id=093f97d0487a494ba1a423c0af169556]) successfully deleted
I20260929 03:01:20.664987 23397 ts_tablet_manager.cc:1918] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:01:20.665192 23397 log.cc:1196] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/9b17f27457aa4cfeaa2b1d1a349164a5
I20260929 03:01:20.665701 23397 ts_tablet_manager.cc:1939] T 9b17f27457aa4cfeaa2b1d1a349164a5 P b2a0147c9fd749678504d5f360795338: Deleting consensus metadata
I20260929 03:01:20.667385 22885 catalog_manager.cc:4841] TS b2a0147c9fd749678504d5f360795338 (127.8.208.67:35347): tablet 9b17f27457aa4cfeaa2b1d1a349164a5 (table test-table [id=093f97d0487a494ba1a423c0af169556]) successfully deleted
I20260929 03:01:20.671252 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.633529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34278 (local address 127.8.208.65:35459)
0929 03:01:20.633743 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:20.633752 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:20.644030 (+ 10278us) server_negotiation.cc:407] Connection header received
0929 03:01:20.644470 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:20.644475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:20.644575 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:20.644667 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:20.649687 (+ 5020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.650867 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.652950 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.653707 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.653803 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.664870 (+ 11067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:20.664884 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:20.664901 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:20.664947 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:20.668460 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:20.669310 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:20.669316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:20.669319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:20.669444 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:20.669844 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:20.669847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:20.669849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:20.670182 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:20.670391 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:20.670785 (+ 394us) server_negotiation.cc:299] Negotiation successful
0929 03:01:20.670942 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":66}
I20260929 03:01:20.671979 23083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:20.762037 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 22960
I20260929 03:01:20.779637 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 23094
Sep 29 03:01:20 dist-test-slave-m1rv krb5kdc[22854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 03:01:20.795648 23368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.780559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.8.208.66:43993)
0929 03:01:20.780760 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:20.780766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:20.780786 (+ 20us) server_negotiation.cc:407] Connection header received
0929 03:01:20.780883 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:20.780888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:20.780978 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:20.781115 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:20.782397 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.783221 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.784476 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.784939 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.785094 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.789750 (+ 4656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:20.789764 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:20.789783 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:20.789825 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:20.792770 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:20.793618 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:20.793625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:20.793627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:20.793766 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:20.794140 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:20.794153 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:20.794155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:20.794496 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:20.794830 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:20.795072 (+ 242us) server_negotiation.cc:299] Negotiation successful
0929 03:01:20.795306 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 03:01:20.796415 23217 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:20.909981 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 23094
I20260929 03:01:20.929567 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 23227
Sep 29 03:01:20 dist-test-slave-m1rv krb5kdc[22854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 03:01:20.945925 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:20.930408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34772 (local address 127.8.208.67:35347)
0929 03:01:20.930591 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:20.930597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:20.930647 (+ 50us) server_negotiation.cc:407] Connection header received
0929 03:01:20.930874 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:20.930879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:20.930963 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:20.931074 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:20.932424 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.933370 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.934917 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:20.935305 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:20.935499 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:20.940119 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:20.940133 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:20.940150 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:20.940189 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:20.942961 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:20.943660 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:20.943667 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:20.943669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:20.943818 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:20.944149 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:20.944152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:20.944154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:20.944489 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:20.944715 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:20.945016 (+ 301us) server_negotiation.cc:299] Negotiation successful
0929 03:01:20.945340 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 03:01:20.946596 23350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:21.037015 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 23227
I20260929 03:01:21.053649 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 22867
I20260929 03:01:21.054802 22929 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:21.165361 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 22867
I20260929 03:01:21.186590 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:46583
--webserver_interface=127.8.208.126
--webserver_port=46435
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:46583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:21.508846 23405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:21.509192 23405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:21.509383 23405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:21.516462 23405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:01:21.516662 23405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:21.516767 23405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:21.516870 23405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:01:21.516954 23405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:01:21.526763 23405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:46583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:46583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=46435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:21.528050 23405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:21.529350 23405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:21.551390 23411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:21.551376 23412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:21.551376 23414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:21.552352 23405 server_base.cc:1034] running on GCE node
I20260929 03:01:21.553290 23405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:21.554958 23405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:21.556195 23405 hybrid_clock.cc:648] HybridClock initialized: now 1790650881556149 us; error 59 us; skew 500 ppm
I20260929 03:01:21.556629 23405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:21 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650881, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:21.560307 23405 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:01:21.561942 23405 webserver.cc:460] Webserver started at http://127.8.208.126:46435/ using document root <none> and password file <none>
I20260929 03:01:21.562665 23405 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:21.562775 23405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:21.568567 23405 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260929 03:01:21.571403 23421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:21.572698 23405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 03:01:21.572889 23405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "015013a3cf5e49439b418017f6eadfe0"
format_stamp: "Formatted at 2026-09-29 03:01:18 on dist-test-slave-m1rv"
server_key: "f62b34461289cd22dbc1506282218e8a"
server_key_iv: "f4a9908fb41f386bd5b048924296b523"
server_key_version: "encryptionkey@0"
I20260929 03:01:21.574007 23405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:21.599957 23405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:21.604749 23405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:21.605027 23405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:21.623761 23405 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:46583
I20260929 03:01:21.623781 23472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:46583 every 8 connection(s)
I20260929 03:01:21.625412 23405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:01:21.627290 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 23405
I20260929 03:01:21.628340 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:35459
--local_ip_for_outbound_sockets=127.8.208.65
--tserver_master_addrs=127.8.208.126:46583
--webserver_port=35591
--webserver_interface=127.8.208.65
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:01:21.635087 23473 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:01:21.639012 23473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Bootstrap starting.
I20260929 03:01:21.658128 23473 log.cc:824] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:21.669484 23473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:01:21.669991 23473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Bootstrap complete.
I20260929 03:01:21.679394 23473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } }
I20260929 03:01:21.680228 23473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015013a3cf5e49439b418017f6eadfe0, State: Initialized, Role: FOLLOWER
I20260929 03:01:21.680841 23473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } }
I20260929 03:01:21.681095 23473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:01:21.681239 23473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:01:21.681437 23473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 1 FOLLOWER]: Advancing to term 2
I20260929 03:01:21.685106 23473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } }
I20260929 03:01:21.685706 23473 leader_election.cc:302] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 015013a3cf5e49439b418017f6eadfe0; no voters:
I20260929 03:01:21.686358 23473 leader_election.cc:288] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 03:01:21.686489 23477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 03:01:21.687597 23477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 2 LEADER]: Becoming Leader. State: Replica: 015013a3cf5e49439b418017f6eadfe0, State: Running, Role: LEADER
I20260929 03:01:21.688199 23477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } }
I20260929 03:01:21.688598 23473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:01:21.690593 23479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 015013a3cf5e49439b418017f6eadfe0. Latest consensus state: current_term: 2 leader_uuid: "015013a3cf5e49439b418017f6eadfe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } } }
I20260929 03:01:21.691398 23479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:21.691061 23478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "015013a3cf5e49439b418017f6eadfe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } } }
I20260929 03:01:21.691782 23478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:21.696165 23483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:01:21.703593 23483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:01:21.704973 23483 catalog_manager.cc:1259] Loaded cluster ID: fd861c2da81844a7964692655b07fecc
I20260929 03:01:21.705134 23483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:01:21.708199 23483 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 03:01:21.726806 23483 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 03:01:21.736266 23483 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:01:21.738178 23483 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Loaded TSK: 0
I20260929 03:01:21.739795 23483 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 03:01:21.999486 23475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:21.999760 23475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:21.999918 23475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:22.008348 23475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:22.008458 23475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:22.008644 23475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:01:22.019269 23475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:35459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=35591
--tserver_master_addrs=127.8.208.126:46583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:22.020536 23475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:22.021905 23475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:22.036994 23503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:22.036972 23501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:22.041451 23500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:22.043056 23475 server_base.cc:1034] running on GCE node
I20260929 03:01:22.043840 23475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:22.044853 23475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:22.046103 23475 hybrid_clock.cc:648] HybridClock initialized: now 1790650882046035 us; error 62 us; skew 500 ppm
I20260929 03:01:22.046443 23475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:22 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650882, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:22.049618 23475 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 03:01:22.051214 23475 webserver.cc:460] Webserver started at http://127.8.208.65:35591/ using document root <none> and password file <none>
I20260929 03:01:22.051919 23475 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:22.052063 23475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:22.057446 23475 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:01:22.060148 23510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:22.061592 23475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:01:22.061756 23475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "a96430697e744447ad89aea3e4ff7a3e"
format_stamp: "Formatted at 2026-09-29 03:01:19 on dist-test-slave-m1rv"
server_key: "ad469d81ee639024837d822109b905a4"
server_key_iv: "b08e1a2da0d69de285a6745c3717b30c"
server_key_version: "encryptionkey@0"
I20260929 03:01:22.062713 23475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:22.083742 23475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:22.088209 23475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:22.088543 23475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:22.089754 23475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:22.091141 23475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:22.091259 23475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:22.091403 23475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:22.091530 23475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:22.125423 23475 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:35459
I20260929 03:01:22.125495 23622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:35459 every 8 connection(s)
I20260929 03:01:22.126869 23475 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:01:22.134719 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 23475
I20260929 03:01:22.135488 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:43993
--local_ip_for_outbound_sockets=127.8.208.66
--tserver_master_addrs=127.8.208.126:46583
--webserver_port=46839
--webserver_interface=127.8.208.66
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:01:22 dist-test-slave-m1rv krb5kdc[22854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650882, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:22.159196 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:22.129638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:59785 (local address 127.8.208.126:46583)
0929 03:01:22.130198 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:22.130210 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:22.130863 (+ 653us) server_negotiation.cc:407] Connection header received
0929 03:01:22.132821 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:22.132843 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:22.133151 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:22.133576 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:22.135303 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:22.136671 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:22.139134 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:22.139779 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:22.139972 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:22.151609 (+ 11637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:22.151617 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:22.151634 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:22.151673 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:22.154648 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:22.155404 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:22.155411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:22.155417 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:22.155587 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:22.156041 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:22.156045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:22.156047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:22.156503 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:22.156730 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:22.157540 (+ 810us) server_negotiation.cc:299] Negotiation successful
0929 03:01:22.157916 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":244,"threads_started":1}
I20260929 03:01:22.159196 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:22.129985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:59785)
0929 03:01:22.130682 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:22.130724 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:22.132332 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:22.133692 (+ 1360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:22.133699 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:22.134270 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:22.135058 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:22.135085 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:22.136833 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:22.136841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:22.138973 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:22.138990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:22.140064 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:22.140073 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:22.140383 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:22.148730 (+ 8347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:22.148761 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:22.151282 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:22.154806 (+ 3524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:22.154816 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:22.154831 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:22.155247 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:22.155708 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:22.155713 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:22.155717 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:22.155906 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:22.156654 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:22.156662 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:22.157008 (+ 346us) client_negotiation.cc:769] Sending connection context
0929 03:01:22.157314 (+ 306us) client_negotiation.cc:240] Negotiation successful
0929 03:01:22.157874 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":271,"threads_started":1}
I20260929 03:01:22.162416 23623 heartbeater.cc:347] Connected to a master server at 127.8.208.126:46583
I20260929 03:01:22.162714 23623 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:22.163432 23623 heartbeater.cc:511] Master 127.8.208.126:46583 requested a full tablet report, sending...
I20260929 03:01:22.165668 23438 ts_manager.cc:194] Registered new tserver with Master: a96430697e744447ad89aea3e4ff7a3e (127.8.208.65:35459)
I20260929 03:01:22.167197 23438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:59785
W20260929 03:01:22.473618 23627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:22.473892 23627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:22.474062 23627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:22.481343 23627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:22.481467 23627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:22.481657 23627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:01:22.491407 23627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:43993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=46839
--tserver_master_addrs=127.8.208.126:46583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:22.492707 23627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:22.494032 23627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:22.507221 23632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:22.507323 23633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:22.507934 23627 server_base.cc:1034] running on GCE node
W20260929 03:01:22.507982 23635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:22.508677 23627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:22.509676 23627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:22.510870 23627 hybrid_clock.cc:648] HybridClock initialized: now 1790650882510810 us; error 50 us; skew 500 ppm
I20260929 03:01:22.511193 23627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:22 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650882, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:22.514266 23627 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 03:01:22.515664 23627 webserver.cc:460] Webserver started at http://127.8.208.66:46839/ using document root <none> and password file <none>
I20260929 03:01:22.516355 23627 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:22.516515 23627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:22.521806 23627 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260929 03:01:22.524431 23642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:22.525898 23627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:01:22.526081 23627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "388947b5ad3b4a72b7a37b33b2104bc2"
format_stamp: "Formatted at 2026-09-29 03:01:19 on dist-test-slave-m1rv"
server_key: "f2d7c80c77a02aad05bdd4e53c3df9c8"
server_key_iv: "a19986706b7007669601687066799984"
server_key_version: "encryptionkey@0"
I20260929 03:01:22.527032 23627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:22.551057 23627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:22.554998 23627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:22.555284 23627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:22.556275 23627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:22.557672 23627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:22.557788 23627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:22.557919 23627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:22.558012 23627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:22.590759 23627 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:43993
I20260929 03:01:22.590843 23754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:43993 every 8 connection(s)
I20260929 03:01:22.592216 23627 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:01:22.599371 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 23627
I20260929 03:01:22.600020 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:35347
--local_ip_for_outbound_sockets=127.8.208.67
--tserver_master_addrs=127.8.208.126:46583
--webserver_port=41381
--webserver_interface=127.8.208.67
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:01:22 dist-test-slave-m1rv krb5kdc[22854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650882, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:22.614650 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:22.594711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:57079 (local address 127.8.208.126:46583)
0929 03:01:22.594897 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:22.594901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:22.596012 (+ 1111us) server_negotiation.cc:407] Connection header received
0929 03:01:22.597834 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:22.597840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:22.597909 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:22.598007 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:22.599682 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:22.600485 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:22.603025 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:22.603439 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:22.603528 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:22.607577 (+ 4049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:22.607584 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:22.607591 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:22.607638 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:22.611238 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:22.611969 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:22.611976 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:22.611982 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:22.612144 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:22.612637 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:22.612645 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:22.612651 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:22.613009 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:22.613165 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:22.613887 (+ 722us) server_negotiation.cc:299] Negotiation successful
0929 03:01:22.614164 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 03:01:22.615164 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:22.595180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:57079)
0929 03:01:22.595848 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:22.595884 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:22.597590 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:22.598213 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:22.598221 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:22.598708 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:22.599460 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:22.599490 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:22.600648 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:22.600657 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:22.602859 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:22.602875 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:22.603663 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:22.603671 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:22.604026 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:22.604759 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:22.604792 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:22.607359 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:22.611383 (+ 4024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:22.611389 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:22.611402 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:22.611813 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:22.612268 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:22.612274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:22.612279 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:22.612513 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:22.613143 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:22.613149 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:22.613508 (+ 359us) client_negotiation.cc:769] Sending connection context
0929 03:01:22.613725 (+ 217us) client_negotiation.cc:240] Negotiation successful
0929 03:01:22.614086 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":268,"threads_started":1}
I20260929 03:01:22.616637 23755 heartbeater.cc:347] Connected to a master server at 127.8.208.126:46583
I20260929 03:01:22.616941 23755 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:22.617671 23755 heartbeater.cc:511] Master 127.8.208.126:46583 requested a full tablet report, sending...
I20260929 03:01:22.619071 23438 ts_manager.cc:194] Registered new tserver with Master: 388947b5ad3b4a72b7a37b33b2104bc2 (127.8.208.66:43993)
I20260929 03:01:22.619853 23438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:57079
W20260929 03:01:22.923800 23758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:22.924062 23758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:22.924232 23758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:22.931361 23758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:22.931494 23758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:22.931690 23758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:01:22.941227 23758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:35347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=41381
--tserver_master_addrs=127.8.208.126:46583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:22.942445 23758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:22.943797 23758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:22.956790 23764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:22.956808 23763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:22.956928 23766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:22.957569 23758 server_base.cc:1034] running on GCE node
I20260929 03:01:22.958140 23758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:22.959016 23758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:22.960244 23758 hybrid_clock.cc:648] HybridClock initialized: now 1790650882960209 us; error 44 us; skew 500 ppm
I20260929 03:01:22.960561 23758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:22 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650882, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:22.963577 23758 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 03:01:22.965073 23758 webserver.cc:460] Webserver started at http://127.8.208.67:41381/ using document root <none> and password file <none>
I20260929 03:01:22.965842 23758 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:22.965991 23758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:22.971176 23758 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260929 03:01:22.973742 23773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:22.975016 23758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:01:22.975162 23758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "b2a0147c9fd749678504d5f360795338"
format_stamp: "Formatted at 2026-09-29 03:01:20 on dist-test-slave-m1rv"
server_key: "c22dc6fb70fd6aeaf1d90b28989b002d"
server_key_iv: "70fa2a7177bbb470d253514559d07887"
server_key_version: "encryptionkey@0"
I20260929 03:01:22.976065 23758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:22.990272 23758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:22.993837 23758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:22.994060 23758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:22.994939 23758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:22.996146 23758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:22.996230 23758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:22.996304 23758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:22.996364 23758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:23.028396 23758 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:35347
I20260929 03:01:23.028484 23885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:35347 every 8 connection(s)
I20260929 03:01:23.029923 23758 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:01:23.035573 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 23758
Sep 29 03:01:23 dist-test-slave-m1rv krb5kdc[22854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650882, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:23.051340 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.036640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.8.208.126:46583)
0929 03:01:23.037386 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:23.037393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:23.037416 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:01:23.037456 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:23.037459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:23.037514 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:23.037620 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:23.039111 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.039841 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.041146 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.041721 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.041860 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.044968 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:23.044973 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:23.044976 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:23.045014 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:23.048243 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:23.049012 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:23.049016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:23.049019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:23.049252 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:23.049677 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:23.049681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:23.049682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:23.050100 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:23.050343 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:23.050658 (+ 315us) server_negotiation.cc:299] Negotiation successful
0929 03:01:23.050807 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":629,"spinlock_wait_cycles":4736,"thread_start_us":502,"threads_started":1}
I20260929 03:01:23.051743 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.032721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:60575 (local address 127.8.208.126:46583)
0929 03:01:23.032904 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:23.032909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:23.034116 (+ 1207us) server_negotiation.cc:407] Connection header received
0929 03:01:23.035927 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:23.035933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:23.036075 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:23.036199 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:23.038108 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.038826 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.040379 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.040807 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.040938 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.045166 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:23.045216 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:23.045221 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:23.045275 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:23.048130 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:23.049006 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:23.049012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:23.049015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:23.049257 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:23.049766 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:23.049772 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:23.049777 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:23.050100 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:23.050252 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:23.051146 (+ 894us) server_negotiation.cc:299] Negotiation successful
0929 03:01:23.051373 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":52,"spinlock_wait_cycles":10752}
I20260929 03:01:23.052862 23888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.033144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:60575)
0929 03:01:23.033952 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:23.034001 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:23.035587 (+ 1586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:23.036464 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:23.036477 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:23.037348 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:23.037930 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:23.037947 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.038984 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.038990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:23.040220 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:23.040232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.041037 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.041045 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:23.041334 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.042057 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:23.042090 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:23.044945 (+ 2855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:23.048275 (+ 3330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:23.048284 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:23.048300 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:23.048890 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:23.049369 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:23.049374 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:23.049378 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:23.049647 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:23.050232 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:23.050240 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:23.050714 (+ 474us) client_negotiation.cc:769] Sending connection context
0929 03:01:23.051021 (+ 307us) client_negotiation.cc:240] Negotiation successful
0929 03:01:23.051442 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":272,"threads_started":1}
I20260929 03:01:23.054375 23886 heartbeater.cc:347] Connected to a master server at 127.8.208.126:46583
I20260929 03:01:23.054675 23886 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:23.055351 23886 heartbeater.cc:511] Master 127.8.208.126:46583 requested a full tablet report, sending...
I20260929 03:01:23.056676 23438 ts_manager.cc:194] Registered new tserver with Master: b2a0147c9fd749678504d5f360795338 (127.8.208.67:35347)
I20260929 03:01:23.057536 23438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:60575
I20260929 03:01:23.066399 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:01:23.072615 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.067301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.8.208.126:46583)
0929 03:01:23.067468 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:23.067474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:23.067932 (+ 458us) server_negotiation.cc:407] Connection header received
0929 03:01:23.068095 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:23.068099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:23.068215 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:23.068328 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:01:23.068675 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.069487 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.070893 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.071256 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.071341 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.071734 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:01:23.072180 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:01:23.072283 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:23.072374 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 03:01:23.072431 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 03:01:23.073515 23438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:01:23.080796 23438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:01:23.103688 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.094320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35347 (local address 127.0.0.1:34780)
0929 03:01:23.095585 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:23.095608 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:23.095700 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:23.096863 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:23.096871 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:23.096907 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:23.097404 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:23.097418 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.099359 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.099367 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:23.100963 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:23.100975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.102617 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.102623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:23.102826 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.102898 (+ 72us) client_negotiation.cc:769] Sending connection context
0929 03:01:23.103033 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 03:01:23.103235 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1137,"mutex_wait_us":89,"thread_start_us":491,"threads_started":1}
I20260929 03:01:23.103847 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.095016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34780 (local address 127.8.208.67:35347)
0929 03:01:23.096271 (+ 1255us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:23.096281 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:23.096313 (+ 32us) server_negotiation.cc:407] Connection header received
0929 03:01:23.096409 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:23.096415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:23.096632 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:23.096855 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:23.097899 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.099208 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.101132 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.102401 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.102562 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.102816 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:23.103325 (+ 509us) server_negotiation.cc:299] Negotiation successful
0929 03:01:23.103485 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":928,"threads_started":1}
I20260929 03:01:23.104628 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.093905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48794 (local address 127.8.208.66:43993)
0929 03:01:23.094486 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:23.094496 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:23.095311 (+ 815us) server_negotiation.cc:407] Connection header received
0929 03:01:23.095740 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:23.095752 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:23.096022 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:23.096188 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:23.097392 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.098899 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.101717 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.103187 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.103335 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.103565 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:23.104097 (+ 532us) server_negotiation.cc:299] Negotiation successful
0929 03:01:23.104259 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":265,"threads_started":1}
I20260929 03:01:23.105423 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.093501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:43993 (local address 127.0.0.1:48794)
0929 03:01:23.095172 (+ 1671us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:23.095217 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:23.095484 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:23.096584 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:23.096591 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:23.096619 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:23.097101 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:23.097121 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.099091 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.099098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:23.101549 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:23.101565 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.103724 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.103730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:23.103893 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.103926 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 03:01:23.104831 (+ 905us) client_negotiation.cc:240] Negotiation successful
0929 03:01:23.104961 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1445,"mutex_wait_us":1,"thread_start_us":1039,"threads_started":1}
I20260929 03:01:23.106135 23900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.095367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35459 (local address 127.0.0.1:34284)
0929 03:01:23.097940 (+ 2573us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:23.097960 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:23.098072 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:23.098807 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:23.098811 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:23.098851 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:23.099262 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:23.099272 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.101230 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.101235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:23.103361 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:23.103378 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.105345 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.105351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:23.105552 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.105583 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 03:01:23.105678 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 03:01:23.105803 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2441,"thread_start_us":2314,"threads_started":1}
I20260929 03:01:23.106555 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.095951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34284 (local address 127.8.208.65:35459)
0929 03:01:23.097516 (+ 1565us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:23.097524 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:23.098181 (+ 657us) server_negotiation.cc:407] Connection header received
0929 03:01:23.098266 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:23.098272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:23.098460 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:23.098614 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:23.099492 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.101052 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.103534 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.105075 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.105258 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.105452 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:23.106074 (+ 622us) server_negotiation.cc:299] Negotiation successful
0929 03:01:23.106231 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1402,"thread_start_us":1271,"threads_started":1}
I20260929 03:01:23.108008 23821 tablet_service.cc:1467] Processing CreateTablet for tablet eb872d936a534ab5ad8ccce7cf9dfd12 (DEFAULT_TABLE table=test-table [id=6a446cb61f60450688ca2e5c9ef3a117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:01:23.108953 23690 tablet_service.cc:1467] Processing CreateTablet for tablet eb872d936a534ab5ad8ccce7cf9dfd12 (DEFAULT_TABLE table=test-table [id=6a446cb61f60450688ca2e5c9ef3a117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:01:23.109552 23821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb872d936a534ab5ad8ccce7cf9dfd12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:23.110111 23690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb872d936a534ab5ad8ccce7cf9dfd12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:23.110065 23558 tablet_service.cc:1467] Processing CreateTablet for tablet eb872d936a534ab5ad8ccce7cf9dfd12 (DEFAULT_TABLE table=test-table [id=6a446cb61f60450688ca2e5c9ef3a117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:01:23.111343 23558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb872d936a534ab5ad8ccce7cf9dfd12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:23.121799 23902 tablet_bootstrap.cc:492] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2: Bootstrap starting.
I20260929 03:01:23.125065 23902 tablet_bootstrap.cc:654] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:23.126387 23902 log.cc:824] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:23.134048 23903 tablet_bootstrap.cc:492] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338: Bootstrap starting.
I20260929 03:01:23.134883 23904 tablet_bootstrap.cc:492] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e: Bootstrap starting.
I20260929 03:01:23.136768 23902 tablet_bootstrap.cc:492] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2: No bootstrap required, opened a new log
I20260929 03:01:23.137089 23902 ts_tablet_manager.cc:1397] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.013s
I20260929 03:01:23.138135 23903 tablet_bootstrap.cc:654] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:23.139568 23903 log.cc:824] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:23.138926 23904 tablet_bootstrap.cc:654] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:23.141710 23904 log.cc:824] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:23.144510 23903 tablet_bootstrap.cc:492] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338: No bootstrap required, opened a new log
I20260929 03:01:23.144807 23903 ts_tablet_manager.cc:1397] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260929 03:01:23.144959 23904 tablet_bootstrap.cc:492] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e: No bootstrap required, opened a new log
I20260929 03:01:23.145252 23904 ts_tablet_manager.cc:1397] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260929 03:01:23.147369 23902 raft_consensus.cc:348] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:23.147815 23902 raft_consensus.cc:374] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:23.147965 23902 raft_consensus.cc:729] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 388947b5ad3b4a72b7a37b33b2104bc2, State: Initialized, Role: FOLLOWER
I20260929 03:01:23.148614 23902 consensus_queue.cc:260] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:23.149976 23755 heartbeater.cc:503] Master 127.8.208.126:46583 was elected leader, sending a full tablet report...
I20260929 03:01:23.150557 23902 ts_tablet_manager.cc:1428] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260929 03:01:23.156045 23903 raft_consensus.cc:348] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:23.156494 23903 raft_consensus.cc:374] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:23.156613 23903 raft_consensus.cc:729] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a0147c9fd749678504d5f360795338, State: Initialized, Role: FOLLOWER
I20260929 03:01:23.156505 23904 raft_consensus.cc:348] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:23.156929 23904 raft_consensus.cc:374] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:23.157037 23904 raft_consensus.cc:729] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96430697e744447ad89aea3e4ff7a3e, State: Initialized, Role: FOLLOWER
I20260929 03:01:23.157253 23903 consensus_queue.cc:260] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:23.157611 23904 consensus_queue.cc:260] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:23.158479 23886 heartbeater.cc:503] Master 127.8.208.126:46583 was elected leader, sending a full tablet report...
I20260929 03:01:23.158761 23623 heartbeater.cc:503] Master 127.8.208.126:46583 was elected leader, sending a full tablet report...
I20260929 03:01:23.159014 23903 ts_tablet_manager.cc:1428] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
I20260929 03:01:23.159312 23904 ts_tablet_manager.cc:1428] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260929 03:01:23.281576 23887 tablet.cc:2369] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:01:23.344624 23756 tablet.cc:2369] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:01:23.379276 23624 tablet.cc:2369] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:01:23.529350 23909 raft_consensus.cc:482] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:01:23.529680 23909 raft_consensus.cc:504] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:23.531440 23909 leader_election.cc:288] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 388947b5ad3b4a72b7a37b33b2104bc2 (127.8.208.66:43993), a96430697e744447ad89aea3e4ff7a3e (127.8.208.65:35459)
I20260929 03:01:23.537328 23888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.531652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:43993 (local address 127.8.208.67:44299)
0929 03:01:23.531795 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:23.531816 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:23.531947 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:23.532304 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:23.532311 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:23.532344 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:23.532568 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:23.532582 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.533723 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.533731 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:23.535439 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:23.535453 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.536628 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.536632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:23.536759 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.536782 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 03:01:23.536892 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 03:01:23.537039 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":26}
I20260929 03:01:23.537322 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.531753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:44299 (local address 127.8.208.66:43993)
0929 03:01:23.531955 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:23.531961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:23.531985 (+ 24us) server_negotiation.cc:407] Connection header received
0929 03:01:23.532070 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:23.532075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:23.532149 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:23.532256 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:23.532793 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.533578 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.535594 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.536342 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.536482 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.536536 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:23.536919 (+ 383us) server_negotiation.cc:299] Negotiation successful
0929 03:01:23.537042 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:01:23.538435 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.532017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35459 (local address 127.8.208.67:43623)
0929 03:01:23.532669 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:23.532687 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:23.532841 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:23.533209 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:23.533214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:23.533238 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:23.533572 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:23.533580 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.534722 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.534728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:23.536322 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:23.536334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.537748 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.537754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:23.537920 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.537957 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 03:01:23.538069 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 03:01:23.538188 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":1,"thread_start_us":316,"threads_started":1}
I20260929 03:01:23.538483 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.532160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:43623 (local address 127.8.208.65:35459)
0929 03:01:23.532720 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:23.532728 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:23.532751 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:01:23.532990 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:23.532995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:23.533072 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:23.533167 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:23.533825 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.534576 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.536485 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.537373 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.537540 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.537601 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:23.538104 (+ 503us) server_negotiation.cc:299] Negotiation successful
0929 03:01:23.538237 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382}
I20260929 03:01:23.538477 23710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb872d936a534ab5ad8ccce7cf9dfd12" candidate_uuid: "b2a0147c9fd749678504d5f360795338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" is_pre_election: true
I20260929 03:01:23.538988 23710 raft_consensus.cc:2454] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2a0147c9fd749678504d5f360795338 in term 0.
I20260929 03:01:23.539691 23777 leader_election.cc:302] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388947b5ad3b4a72b7a37b33b2104bc2, b2a0147c9fd749678504d5f360795338; no voters:
I20260929 03:01:23.539695 23578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb872d936a534ab5ad8ccce7cf9dfd12" candidate_uuid: "b2a0147c9fd749678504d5f360795338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96430697e744447ad89aea3e4ff7a3e" is_pre_election: true
I20260929 03:01:23.540155 23909 raft_consensus.cc:2781] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:01:23.540232 23578 raft_consensus.cc:2454] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2a0147c9fd749678504d5f360795338 in term 0.
I20260929 03:01:23.540304 23909 raft_consensus.cc:482] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:01:23.540410 23909 raft_consensus.cc:3037] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:23.542757 23909 raft_consensus.cc:504] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:23.543349 23909 leader_election.cc:288] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [CANDIDATE]: Term 1 election: Requested vote from peers 388947b5ad3b4a72b7a37b33b2104bc2 (127.8.208.66:43993), a96430697e744447ad89aea3e4ff7a3e (127.8.208.65:35459)
I20260929 03:01:23.543691 23710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb872d936a534ab5ad8ccce7cf9dfd12" candidate_uuid: "b2a0147c9fd749678504d5f360795338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "388947b5ad3b4a72b7a37b33b2104bc2"
I20260929 03:01:23.543790 23578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb872d936a534ab5ad8ccce7cf9dfd12" candidate_uuid: "b2a0147c9fd749678504d5f360795338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96430697e744447ad89aea3e4ff7a3e"
I20260929 03:01:23.543879 23710 raft_consensus.cc:3037] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:23.544021 23578 raft_consensus.cc:3037] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:23.546219 23578 raft_consensus.cc:2454] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2a0147c9fd749678504d5f360795338 in term 1.
I20260929 03:01:23.546229 23710 raft_consensus.cc:2454] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2a0147c9fd749678504d5f360795338 in term 1.
I20260929 03:01:23.546613 23777 leader_election.cc:302] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388947b5ad3b4a72b7a37b33b2104bc2, b2a0147c9fd749678504d5f360795338; no voters:
I20260929 03:01:23.546911 23909 raft_consensus.cc:2781] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:01:23.547348 23909 raft_consensus.cc:686] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 1 LEADER]: Becoming Leader. State: Replica: b2a0147c9fd749678504d5f360795338, State: Running, Role: LEADER
I20260929 03:01:23.547799 23909 consensus_queue.cc:237] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } }
I20260929 03:01:23.553050 23437 catalog_manager.cc:5495] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2a0147c9fd749678504d5f360795338 (127.8.208.67). New cstate: current_term: 1 leader_uuid: "b2a0147c9fd749678504d5f360795338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2a0147c9fd749678504d5f360795338" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35347 } health_report { overall_health: HEALTHY } } }
I20260929 03:01:23.675093 23916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.669715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34784 (local address 127.8.208.67:35347)
0929 03:01:23.670479 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:23.670483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:23.670499 (+ 16us) server_negotiation.cc:407] Connection header received
0929 03:01:23.670586 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:23.670589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:23.670662 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:23.670797 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:01:23.671149 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.671790 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.673145 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.673591 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.673692 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.674126 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:01:23.674607 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:01:23.674720 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:23.674798 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 03:01:23.674861 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":559,"threads_started":1}
I20260929 03:01:23.682761 23710 raft_consensus.cc:1261] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 1 FOLLOWER]: Refusing update from remote peer b2a0147c9fd749678504d5f360795338: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:01:23.682762 23578 raft_consensus.cc:1261] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e [term 1 FOLLOWER]: Refusing update from remote peer b2a0147c9fd749678504d5f360795338: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:01:23.683693 23912 consensus_queue.cc:1035] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "388947b5ad3b4a72b7a37b33b2104bc2" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:01:23.684024 23909 consensus_queue.cc:1035] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a96430697e744447ad89aea3e4ff7a3e" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:01:23.694537 23917 mvcc.cc:205] Tried to move back new op lower bound from 7334506019555393536 to 7334506019015475200. Current Snapshot: MvccSnapshot[applied={T|T < 7334506019555393536}]
I20260929 03:01:23.721458 23437 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:01:23.721781 23437 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:01:23.725224 23437 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Sending DeleteTablet for 3 replicas of tablet eb872d936a534ab5ad8ccce7cf9dfd12
I20260929 03:01:23.726416 23821 tablet_service.cc:1514] Processing DeleteTablet for tablet eb872d936a534ab5ad8ccce7cf9dfd12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:01:23 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:34780
I20260929 03:01:23.726496 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 23475
I20260929 03:01:23.726686 23558 tablet_service.cc:1514] Processing DeleteTablet for tablet eb872d936a534ab5ad8ccce7cf9dfd12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:01:23 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:34284
I20260929 03:01:23.727223 23690 tablet_service.cc:1514] Processing DeleteTablet for tablet eb872d936a534ab5ad8ccce7cf9dfd12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:01:23 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:48794
I20260929 03:01:23.741909 23930 tablet_replica.cc:330] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e: stopping tablet replica
I20260929 03:01:23.742492 23930 raft_consensus.cc:2229] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:01:23.743086 23930 raft_consensus.cc:2258] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:01:23.747248 23930 ts_tablet_manager.cc:1905] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:01:23.755048 23931 tablet_replica.cc:330] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2: stopping tablet replica
I20260929 03:01:23.755148 23930 ts_tablet_manager.cc:1918] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:01:23.755333 23930 log.cc:1196] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/eb872d936a534ab5ad8ccce7cf9dfd12
I20260929 03:01:23.755733 23931 raft_consensus.cc:2229] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:01:23.755890 23930 ts_tablet_manager.cc:1939] T eb872d936a534ab5ad8ccce7cf9dfd12 P a96430697e744447ad89aea3e4ff7a3e: Deleting consensus metadata
I20260929 03:01:23.756366 23931 raft_consensus.cc:2258] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:01:23.757877 23425 catalog_manager.cc:4841] TS a96430697e744447ad89aea3e4ff7a3e (127.8.208.65:35459): tablet eb872d936a534ab5ad8ccce7cf9dfd12 (table test-table [id=6a446cb61f60450688ca2e5c9ef3a117]) successfully deleted
I20260929 03:01:23.758962 23929 tablet_replica.cc:330] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338: stopping tablet replica
I20260929 03:01:23.759451 23929 raft_consensus.cc:2229] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:01:23.760190 23929 raft_consensus.cc:2258] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:01:23.760388 23931 ts_tablet_manager.cc:1905] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:01:23.762177 23929 ts_tablet_manager.cc:1905] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:01:23.767498 23931 ts_tablet_manager.cc:1918] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:01:23.767583 23929 ts_tablet_manager.cc:1918] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:01:23.767673 23931 log.cc:1196] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/eb872d936a534ab5ad8ccce7cf9dfd12
I20260929 03:01:23.767791 23929 log.cc:1196] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/eb872d936a534ab5ad8ccce7cf9dfd12
I20260929 03:01:23.768286 23929 ts_tablet_manager.cc:1939] T eb872d936a534ab5ad8ccce7cf9dfd12 P b2a0147c9fd749678504d5f360795338: Deleting consensus metadata
I20260929 03:01:23.768316 23931 ts_tablet_manager.cc:1939] T eb872d936a534ab5ad8ccce7cf9dfd12 P 388947b5ad3b4a72b7a37b33b2104bc2: Deleting consensus metadata
I20260929 03:01:23.770049 23425 catalog_manager.cc:4841] TS 388947b5ad3b4a72b7a37b33b2104bc2 (127.8.208.66:43993): tablet eb872d936a534ab5ad8ccce7cf9dfd12 (table test-table [id=6a446cb61f60450688ca2e5c9ef3a117]) successfully deleted
I20260929 03:01:23.770049 23424 catalog_manager.cc:4841] TS b2a0147c9fd749678504d5f360795338 (127.8.208.67:35347): tablet eb872d936a534ab5ad8ccce7cf9dfd12 (table test-table [id=6a446cb61f60450688ca2e5c9ef3a117]) successfully deleted
I20260929 03:01:23.778203 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.728701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34288 (local address 127.8.208.65:35459)
0929 03:01:23.728925 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:23.728934 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:23.751404 (+ 22470us) server_negotiation.cc:407] Connection header received
0929 03:01:23.751925 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:23.751931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:23.752023 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:23.752101 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:23.756603 (+ 4502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.757681 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.760529 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.761188 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.761316 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.771325 (+ 10009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:23.771343 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:23.771363 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:23.771410 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:23.775196 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:23.775902 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:23.775911 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:23.775915 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:23.776074 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:23.776423 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:23.776428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:23.776432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:23.776946 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:23.777282 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:23.777623 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 03:01:23.777849 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 03:01:23.778967 23618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:23.866941 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 23475
I20260929 03:01:23.887557 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 23627
I20260929 03:01:23.902971 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:23.888468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48796 (local address 127.8.208.66:43993)
0929 03:01:23.888684 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:23.888691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:23.888737 (+ 46us) server_negotiation.cc:407] Connection header received
0929 03:01:23.888828 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:23.888834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:23.888927 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:23.889036 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:23.890464 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.891633 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.892915 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:23.893633 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:23.893763 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:23.896349 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:23.896367 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:23.896387 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:23.896434 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:23.900146 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:23.900791 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:23.900802 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:23.900806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:23.900972 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:23.901364 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:23.901369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:23.901372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:23.901832 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:23.902146 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:23.902351 (+ 205us) server_negotiation.cc:299] Negotiation successful
0929 03:01:23.902587 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 03:01:23.903748 23750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:23.992467 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 23627
I20260929 03:01:24.010087 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 23758
I20260929 03:01:24.026468 23916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:24.010938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34798 (local address 127.8.208.67:35347)
0929 03:01:24.011127 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:24.011134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:24.011156 (+ 22us) server_negotiation.cc:407] Connection header received
0929 03:01:24.011338 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:24.011343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:24.011438 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:24.011557 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:24.012906 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:24.014176 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:24.015535 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:24.016228 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:24.016362 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:24.019030 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:24.019048 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:24.019075 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:24.019117 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:24.023397 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:24.024060 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:24.024074 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:24.024078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:24.024261 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:24.024619 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:24.024624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:24.024628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:24.025125 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:24.025461 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:24.025803 (+ 342us) server_negotiation.cc:299] Negotiation successful
0929 03:01:24.026074 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 03:01:24.027208 23881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:24.149305 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 23758
I20260929 03:01:24.170332 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 23405
I20260929 03:01:24.171628 23468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:24.286574 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 23405
I20260929 03:01:24.308099 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:46583
--webserver_interface=127.8.208.126
--webserver_port=46435
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:46583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:24.628175 23937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:24.628443 23937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:24.628616 23937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:24.636044 23937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:01:24.636168 23937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:24.636248 23937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:24.636341 23937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:01:24.636423 23937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:01:24.646148 23937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:46583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:46583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=46435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:24.647554 23937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:24.648906 23937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:24.660594 23943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:24.660621 23942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:24.660863 23945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:24.661363 23937 server_base.cc:1034] running on GCE node
I20260929 03:01:24.662147 23937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:24.663564 23937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:24.664799 23937 hybrid_clock.cc:648] HybridClock initialized: now 1790650884664737 us; error 46 us; skew 500 ppm
I20260929 03:01:24.665145 23937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:24 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650884, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:24.668179 23937 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:01:24.669701 23937 webserver.cc:460] Webserver started at http://127.8.208.126:46435/ using document root <none> and password file <none>
I20260929 03:01:24.670333 23937 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:24.670429 23937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:24.675724 23937 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 03:01:24.678368 23952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:24.679636 23937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:01:24.679766 23937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "015013a3cf5e49439b418017f6eadfe0"
format_stamp: "Formatted at 2026-09-29 03:01:18 on dist-test-slave-m1rv"
server_key: "f62b34461289cd22dbc1506282218e8a"
server_key_iv: "f4a9908fb41f386bd5b048924296b523"
server_key_version: "encryptionkey@0"
I20260929 03:01:24.680688 23937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:24.693819 23937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:24.697618 23937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:24.697829 23937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:24.715993 23937 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:46583
I20260929 03:01:24.716028 24003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:46583 every 8 connection(s)
I20260929 03:01:24.717653 23937 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:01:24.719640 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 23937
I20260929 03:01:24.720631 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:35459
--local_ip_for_outbound_sockets=127.8.208.65
--tserver_master_addrs=127.8.208.126:46583
--webserver_port=35591
--webserver_interface=127.8.208.65
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:01:24.727289 24004 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:01:24.731300 24004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Bootstrap starting.
I20260929 03:01:24.752079 24004 log.cc:824] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:24.769089 24004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:01:24.769716 24004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0: Bootstrap complete.
I20260929 03:01:24.778988 24004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } }
I20260929 03:01:24.779865 24004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015013a3cf5e49439b418017f6eadfe0, State: Initialized, Role: FOLLOWER
I20260929 03:01:24.780490 24004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } }
I20260929 03:01:24.780719 24004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:01:24.780834 24004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:01:24.781033 24004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 2 FOLLOWER]: Advancing to term 3
I20260929 03:01:24.785068 24004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } }
I20260929 03:01:24.785595 24004 leader_election.cc:302] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 015013a3cf5e49439b418017f6eadfe0; no voters:
I20260929 03:01:24.786286 24004 leader_election.cc:288] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 03:01:24.786422 24008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 03:01:24.787562 24008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [term 3 LEADER]: Becoming Leader. State: Replica: 015013a3cf5e49439b418017f6eadfe0, State: Running, Role: LEADER
I20260929 03:01:24.788172 24008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } }
I20260929 03:01:24.788390 24004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:01:24.790700 24010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 015013a3cf5e49439b418017f6eadfe0. Latest consensus state: current_term: 3 leader_uuid: "015013a3cf5e49439b418017f6eadfe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } } }
I20260929 03:01:24.791083 24010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:24.791628 24009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "015013a3cf5e49439b418017f6eadfe0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015013a3cf5e49439b418017f6eadfe0" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46583 } } }
I20260929 03:01:24.791889 24009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 015013a3cf5e49439b418017f6eadfe0 [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:24.801923 24014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:01:24.810815 24014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:01:24.812248 24014 catalog_manager.cc:1259] Loaded cluster ID: fd861c2da81844a7964692655b07fecc
I20260929 03:01:24.812403 24014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 03:01:24.814357 24014 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f65709f29ad google::LogMessage::Fail() at ??:0
@ 0x7f65709f588a google::LogMessage::SendToLog() at ??:0
@ 0x7f65709f24de google::LogMessage::Flush() at ??:0
@ 0x7f65709f3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6586425512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f6586422976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f65864aa610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f65864aa439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f657363e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f657364191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f6573641799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f657361ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f65768f66db start_thread at ??:0
@ 0x7f656f88c61f clone at ??:0
W20260929 03:01:25.147113 24006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:25.147419 24006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:25.147526 24006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:25.155656 24006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:25.155776 24006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:25.155910 24006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:01:25.166988 24006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:35459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=35591
--tserver_master_addrs=127.8.208.126:46583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:25.168340 24006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:25.169754 24006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:25.183025 24045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:25.183198 24044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:25.183579 24047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:25.184372 24006 server_base.cc:1034] running on GCE node
I20260929 03:01:25.185076 24006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:25.186117 24006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:25.187336 24006 hybrid_clock.cc:648] HybridClock initialized: now 1790650885187295 us; error 32 us; skew 500 ppm
I20260929 03:01:25.187742 24006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:25 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650885, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:25.191334 24006 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 03:01:25.192849 24006 webserver.cc:460] Webserver started at http://127.8.208.65:35591/ using document root <none> and password file <none>
I20260929 03:01:25.193638 24006 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:25.193748 24006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:25.199441 24006 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260929 03:01:25.202246 24054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:25.203653 24006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:01:25.203841 24006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "a96430697e744447ad89aea3e4ff7a3e"
format_stamp: "Formatted at 2026-09-29 03:01:19 on dist-test-slave-m1rv"
server_key: "ad469d81ee639024837d822109b905a4"
server_key_iv: "b08e1a2da0d69de285a6745c3717b30c"
server_key_version: "encryptionkey@0"
I20260929 03:01:25.204908 24006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:25.229748 24006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:25.234634 24006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:25.234941 24006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:25.235994 24006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:25.237421 24006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:25.237556 24006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:25.237696 24006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:25.237821 24006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:25.275439 24006 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:35459
I20260929 03:01:25.275521 24166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:35459 every 8 connection(s)
I20260929 03:01:25.277026 24006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:01:25.282007 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:25.279995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:52473)
0929 03:01:25.280756 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:25.281101 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":305,"threads_started":1}
W20260929 03:01:25.282661 24167 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:46583 (0 consecutive failures): Network error: Failed to ping master at 127.8.208.126:46583: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
I20260929 03:01:25.283490 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:25.283155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:47525)
0929 03:01:25.283308 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:25.283384 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 03:01:25.284413 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:25.284091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:44561)
0929 03:01:25.284216 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:25.284291 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 03:01:25.284721 24167 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 03:01:25.286868 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 24006
I20260929 03:01:25.287480 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:43993
--local_ip_for_outbound_sockets=127.8.208.66
--tserver_master_addrs=127.8.208.126:46583
--webserver_port=46839
--webserver_interface=127.8.208.66
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:25.613642 24170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:25.613920 24170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:25.614014 24170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:25.621048 24170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:25.621148 24170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:25.621342 24170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:01:25.630774 24170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:43993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=46839
--tserver_master_addrs=127.8.208.126:46583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:25.631948 24170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:25.633421 24170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:25.646461 24175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:25.646535 24178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:25.646492 24176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:25.647367 24170 server_base.cc:1034] running on GCE node
I20260929 03:01:25.647915 24170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:25.648834 24170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:25.650025 24170 hybrid_clock.cc:648] HybridClock initialized: now 1790650885649967 us; error 58 us; skew 500 ppm
I20260929 03:01:25.650357 24170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:25 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650885, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:25.653501 24170 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 03:01:25.654924 24170 webserver.cc:460] Webserver started at http://127.8.208.66:46839/ using document root <none> and password file <none>
I20260929 03:01:25.655608 24170 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:25.655745 24170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:25.660961 24170 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:01:25.663530 24185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:25.664857 24170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:01:25.665027 24170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "388947b5ad3b4a72b7a37b33b2104bc2"
format_stamp: "Formatted at 2026-09-29 03:01:19 on dist-test-slave-m1rv"
server_key: "f2d7c80c77a02aad05bdd4e53c3df9c8"
server_key_iv: "a19986706b7007669601687066799984"
server_key_version: "encryptionkey@0"
I20260929 03:01:25.666072 24170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:25.678392 24170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:25.682098 24170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:25.682384 24170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:25.683290 24170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:25.684526 24170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:25.684677 24170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:25.684813 24170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:25.684906 24170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:25.718056 24170 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:43993
I20260929 03:01:25.718142 24297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:43993 every 8 connection(s)
I20260929 03:01:25.719563 24170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:01:25.724496 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:25.722712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:36065)
0929 03:01:25.723413 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:25.723658 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":265,"threads_started":1}
W20260929 03:01:25.725283 24298 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:46583 (0 consecutive failures): Network error: Failed to ping master at 127.8.208.126:46583: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
I20260929 03:01:25.726082 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:25.725777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:39473)
0929 03:01:25.725911 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:25.725978 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 03:01:25.726886 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:25.726600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:45175)
0929 03:01:25.726718 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:25.726786 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 03:01:25.727167 24298 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 03:01:25.729830 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 24170
I20260929 03:01:25.730443 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:35347
--local_ip_for_outbound_sockets=127.8.208.67
--tserver_master_addrs=127.8.208.126:46583
--webserver_port=41381
--webserver_interface=127.8.208.67
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:26.054706 24301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:26.054996 24301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:26.055192 24301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:26.062387 24301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:26.062525 24301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:26.062723 24301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:01:26.072527 24301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:35347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=41381
--tserver_master_addrs=127.8.208.126:46583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:26.073901 24301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:26.075302 24301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:26.088431 24307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:26.088403 24309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:26.088518 24306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:26.089277 24301 server_base.cc:1034] running on GCE node
I20260929 03:01:26.089811 24301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:26.090695 24301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:26.091897 24301 hybrid_clock.cc:648] HybridClock initialized: now 1790650886091831 us; error 56 us; skew 500 ppm
I20260929 03:01:26.092181 24301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:26 dist-test-slave-m1rv krb5kdc[22854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650886, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:26.095553 24301 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 03:01:26.097074 24301 webserver.cc:460] Webserver started at http://127.8.208.67:41381/ using document root <none> and password file <none>
I20260929 03:01:26.097797 24301 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:26.097931 24301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:26.103127 24301 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 03:01:26.105693 24316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:26.106968 24301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 03:01:26.107147 24301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "b2a0147c9fd749678504d5f360795338"
format_stamp: "Formatted at 2026-09-29 03:01:20 on dist-test-slave-m1rv"
server_key: "c22dc6fb70fd6aeaf1d90b28989b002d"
server_key_iv: "70fa2a7177bbb470d253514559d07887"
server_key_version: "encryptionkey@0"
I20260929 03:01:26.108108 24301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:26.143262 24301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:26.147390 24301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:26.147673 24301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:26.148693 24301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:26.150017 24301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:26.150147 24301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:26.150287 24301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:26.150377 24301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:26.183849 24301 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:35347
I20260929 03:01:26.183938 24428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:35347 every 8 connection(s)
I20260929 03:01:26.185415 24301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:01:26.190166 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:26.188317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:48653)
0929 03:01:26.189030 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:26.189340 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":266,"threads_started":1}
W20260929 03:01:26.190842 24429 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:46583 (0 consecutive failures): Network error: Failed to ping master at 127.8.208.126:46583: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
I20260929 03:01:26.191622 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:26.191315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:57617)
0929 03:01:26.191454 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:26.191521 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260929 03:01:26.192402 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:26.192117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:50007)
0929 03:01:26.192234 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:26.192301 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 03:01:26.192638 24429 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 03:01:26.194698 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 24301
I20260929 03:01:26.286000 24433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:26.285250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:39329)
0929 03:01:26.285680 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:26.285764 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":204,"threads_started":1}
I20260929 03:01:26.728474 24434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:26.727685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:38309)
0929 03:01:26.728141 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:26.728243 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":225,"threads_started":1}
I20260929 03:01:27.193953 24435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:27.193217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:39507)
0929 03:01:27.193646 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:27.193730 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":202,"threads_started":1}
I20260929 03:01:27.287588 24436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:27.286892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:34557)
0929 03:01:27.287339 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:27.287419 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":226,"threads_started":1}
I20260929 03:01:27.730135 24437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:27.729435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:47247)
0929 03:01:27.729876 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:27.729956 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":229,"threads_started":1}
I20260929 03:01:28.195510 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:28.194829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:55581)
0929 03:01:28.195266 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:28.195345 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":238,"threads_started":1}
I20260929 03:01:28.289312 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:28.288531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:36713)
0929 03:01:28.288979 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:28.289075 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":228,"threads_started":1}
I20260929 03:01:28.731828 24440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:28.731098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:36257)
0929 03:01:28.731568 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:28.731656 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":239,"threads_started":1}
I20260929 03:01:29.197402 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:29.196640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:34607)
0929 03:01:29.197088 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:29.197228 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":242,"threads_started":1}
I20260929 03:01:29.290939 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:29.290234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:44187)
0929 03:01:29.290690 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:29.290773 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":238,"threads_started":1}
I20260929 03:01:29.733772 24443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:29.732850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:46649)
0929 03:01:29.733395 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:29.733547 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":238,"threads_started":1}
I20260929 03:01:30.198901 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:30.198227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:57645)
0929 03:01:30.198656 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:30.198737 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":221,"threads_started":1}
I20260929 03:01:30.292526 24445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:30.291817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:40209)
0929 03:01:30.292264 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:30.292353 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":225,"threads_started":1}
I20260929 03:01:30.735414 24446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:30.734709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:32905)
0929 03:01:30.735165 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:30.735246 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":235,"threads_started":1}
I20260929 03:01:31.200609 24447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:31.199841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:59747)
0929 03:01:31.200358 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:31.200441 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":303,"threads_started":1}
I20260929 03:01:31.294128 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:31.293446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:55091)
0929 03:01:31.293867 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:31.293956 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":221,"threads_started":1}
I20260929 03:01:31.737138 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:31.736401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:52249)
0929 03:01:31.736877 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:31.736968 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":260,"threads_started":1}
I20260929 03:01:32.202293 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:32.201558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:41743)
0929 03:01:32.202050 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:32.202137 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":279,"threads_started":1}
I20260929 03:01:32.296140 24451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:32.295045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:45525)
0929 03:01:32.295662 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:32.295804 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":267,"threads_started":1}
I20260929 03:01:32.738896 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:32.738120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:46441)
0929 03:01:32.738597 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:32.738678 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":219,"threads_started":1}
I20260929 03:01:33.204046 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:33.203240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:51857)
0929 03:01:33.203725 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:33.203815 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":209,"threads_started":1}
I20260929 03:01:33.297874 24454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:33.297160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:52341)
0929 03:01:33.297642 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:33.297720 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":224,"threads_started":1}
I20260929 03:01:33.740588 24455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:33.739896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:57377)
0929 03:01:33.740334 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:33.740418 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":224,"threads_started":1}
I20260929 03:01:34.205753 24456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:34.205042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:52085)
0929 03:01:34.205507 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:34.205601 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":266,"threads_started":1}
I20260929 03:01:34.299453 24457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:34.298686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:37875)
0929 03:01:34.299199 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:34.299280 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":240,"threads_started":1}
I20260929 03:01:34.742192 24458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:34.741502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:56961)
0929 03:01:34.741940 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:34.742021 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":217,"threads_started":1}
I20260929 03:01:35.207468 24459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:35.206733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:49255)
0929 03:01:35.207196 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:35.207300 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":217,"threads_started":1}
I20260929 03:01:35.301527 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:35.300380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:58909)
0929 03:01:35.301291 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:35.301369 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":768,"thread_start_us":610,"threads_started":1}
I20260929 03:01:35.743978 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:35.743241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:57027)
0929 03:01:35.743738 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:35.743815 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":301,"threads_started":1}
I20260929 03:01:36.209153 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:36.208329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:53111)
0929 03:01:36.208915 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:36.208995 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":271,"threads_started":1}
I20260929 03:01:36.303076 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:36.302324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:33495)
0929 03:01:36.302797 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:36.302900 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":232,"threads_started":1}
I20260929 03:01:36.745959 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:36.745046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:57013)
0929 03:01:36.745678 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:36.745766 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":305,"threads_started":1}
I20260929 03:01:37.210779 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:37.210112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:52485)
0929 03:01:37.210518 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:37.210607 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":203,"threads_started":1}
I20260929 03:01:37.304924 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:37.304260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:45485)
0929 03:01:37.304686 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:37.304775 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":233,"threads_started":1}
I20260929 03:01:37.747589 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:37.746893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:41821)
0929 03:01:37.747345 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:37.747431 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":230,"threads_started":1}
I20260929 03:01:38.212548 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:38.211698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:52211)
0929 03:01:38.212260 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:38.212341 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":326,"threads_started":1}
I20260929 03:01:38.306628 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:38.305897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:53561)
0929 03:01:38.306324 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:38.306445 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":232,"threads_started":1}
I20260929 03:01:38.749516 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:38.748702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:34559)
0929 03:01:38.749238 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:38.749336 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":263,"threads_started":1}
I20260929 03:01:39.214303 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:39.213461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:58511)
0929 03:01:39.213927 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:39.214025 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":234,"threads_started":1}
I20260929 03:01:39.308279 24472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:39.307509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:51409)
0929 03:01:39.307929 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:39.308012 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":211,"threads_started":1}
I20260929 03:01:39.751137 24473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:39.750454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:40995)
0929 03:01:39.750877 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:39.750958 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":204,"threads_started":1}
I20260929 03:01:40.215947 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:40.215122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:60981)
0929 03:01:40.215672 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:40.215769 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":344,"threads_started":1}
I20260929 03:01:40.310174 24475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:40.309480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:42351)
0929 03:01:40.309908 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:40.310005 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":211,"threads_started":1}
I20260929 03:01:40.753232 24476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:40.752413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.66:33419)
0929 03:01:40.752880 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:40.752981 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":242,"threads_started":1}
I20260929 03:01:41.200101 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 24006
I20260929 03:01:41.217823 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:41.217103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.67:59533)
0929 03:01:41.217565 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:41.217651 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":247,"threads_started":1}
I20260929 03:01:41.224997 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:41.201250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55044 (local address 127.8.208.65:35459)
0929 03:01:41.201910 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:41.201924 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:41.201961 (+ 37us) server_negotiation.cc:407] Connection header received
0929 03:01:41.204161 (+ 2200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:41.204171 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:41.204623 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:41.204922 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:41.206344 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:41.207549 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:41.208961 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:41.209776 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:41.209997 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:41.218936 (+ 8939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:41.218943 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:41.218961 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:41.218998 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:41.222108 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:41.222777 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:41.222784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:41.222790 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:41.222945 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:41.223273 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:41.223277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:41.223279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:41.223795 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:41.224017 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:41.224391 (+ 374us) server_negotiation.cc:299] Negotiation successful
0929 03:01:41.224728 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":358,"threads_started":1}
I20260929 03:01:41.226579 24162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:41.313261 24480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:41.312076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46583 (local address 127.8.208.65:57571)
0929 03:01:41.312718 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:41.312921 (+ 203us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":313,"threads_started":1}
I20260929 03:01:41.313827 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 24006
I20260929 03:01:41.330752 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 24170
I20260929 03:01:41.348533 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:41.331666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.8.208.66:43993)
0929 03:01:41.332244 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:41.332259 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:41.332295 (+ 36us) server_negotiation.cc:407] Connection header received
0929 03:01:41.333993 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:41.334002 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:41.334429 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:41.334804 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:41.336296 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:41.337617 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:41.338854 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:41.339511 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:41.339722 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:41.342516 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:41.342524 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:41.342539 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:41.342585 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:41.345318 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:41.345997 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:41.346007 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:41.346020 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:41.346183 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:41.346577 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:41.346582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:41.346585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:41.347069 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:41.347374 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:41.347845 (+ 471us) server_negotiation.cc:299] Negotiation successful
0929 03:01:41.348244 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":299,"threads_started":1}
I20260929 03:01:41.349927 24293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:41.433446 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 24170
I20260929 03:01:41.453083 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 24301
I20260929 03:01:41.471385 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:41.454140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58366 (local address 127.8.208.67:35347)
0929 03:01:41.454769 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:41.454782 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:41.454823 (+ 41us) server_negotiation.cc:407] Connection header received
0929 03:01:41.456470 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:41.456479 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:41.456924 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:41.457347 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:41.458841 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:41.460028 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:41.461386 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:41.462047 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:41.462251 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:41.465051 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:41.465059 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:41.465076 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:41.465132 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:41.468048 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:41.468722 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:41.468735 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:41.468741 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:41.468904 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:41.469361 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:41.469366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:41.469371 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:41.469893 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:41.470201 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:41.470685 (+ 484us) server_negotiation.cc:299] Negotiation successful
0929 03:01:41.471095 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":305,"threads_started":1}
I20260929 03:01:41.472854 24424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:41.556489 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 24301
2026-09-29T03:01:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24730 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:01:41 dist-test-slave-m1rv krb5kdc[24487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:01:41 dist-test-slave-m1rv krb5kdc[24487](info): set up 2 sockets
Sep 29 03:01:41 dist-test-slave-m1rv krb5kdc[24487](info): commencing operation
krb5kdc: starting...
W20260929 03:01:42.702011 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:01:42 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:01:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:01:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:01:42.928818 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37897
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:37897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:43.248452 24500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:43.248739 24500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:43.248915 24500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:43.256264 24500 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 03:01:43.256405 24500 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 03:01:43.256500 24500 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 03:01:43.256615 24500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:01:43.256716 24500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:43.256817 24500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:43.256908 24500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:01:43.256986 24500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:01:43.266620 24500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:37897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:43.267887 24500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:43.269246 24500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:43.281306 24505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:43.281351 24508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:43.281315 24506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:43.282151 24500 server_base.cc:1034] running on GCE node
I20260929 03:01:43.282925 24500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:43.284288 24500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:43.285533 24500 hybrid_clock.cc:648] HybridClock initialized: now 1790650903285457 us; error 54 us; skew 500 ppm
I20260929 03:01:43.285856 24500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:43 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650903, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:43.289005 24500 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:01:43.290443 24500 webserver.cc:460] Webserver started at http://127.8.208.126:38337/ using document root <none> and password file <none>
I20260929 03:01:43.291108 24500 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:43.291236 24500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:43.291549 24500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:01:43.294484 24500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "76455a2e30954eb5a0e057e624da58c9"
format_stamp: "Formatted at 2026-09-29 03:01:43 on dist-test-slave-m1rv"
server_key: "139a53459a770f13f2455028582f387b"
server_key_iv: "e45ac597e02bbf88cf771345701294fa"
server_key_version: "encryptionkey@0"
I20260929 03:01:43.295241 24500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "76455a2e30954eb5a0e057e624da58c9"
format_stamp: "Formatted at 2026-09-29 03:01:43 on dist-test-slave-m1rv"
server_key: "139a53459a770f13f2455028582f387b"
server_key_iv: "e45ac597e02bbf88cf771345701294fa"
server_key_version: "encryptionkey@0"
I20260929 03:01:43.300108 24500 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260929 03:01:43.303399 24515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:43.304724 24500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 03:01:43.304944 24500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "76455a2e30954eb5a0e057e624da58c9"
format_stamp: "Formatted at 2026-09-29 03:01:43 on dist-test-slave-m1rv"
server_key: "139a53459a770f13f2455028582f387b"
server_key_iv: "e45ac597e02bbf88cf771345701294fa"
server_key_version: "encryptionkey@0"
I20260929 03:01:43.305248 24500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:43.330662 24500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:43.334338 24500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:43.334573 24500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:43.352576 24500 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:37897
I20260929 03:01:43.352617 24566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:37897 every 8 connection(s)
I20260929 03:01:43.353894 24500 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:01:43.357542 24567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:43.360371 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 24500
I20260929 03:01:43.360592 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:01:43.361444 9025 external_mini_cluster.cc:1428] Setting key 39b0796fb05d2539d86f7a0272051251
I20260929 03:01:43.370054 24567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Bootstrap starting.
I20260929 03:01:43.373844 24567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:43.375159 24567 log.cc:824] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Log is configured to *not* fsync() on all Append() calls
Sep 29 03:01:43 dist-test-slave-m1rv krb5kdc[24487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:43.378324 24567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: No bootstrap required, opened a new log
I20260929 03:01:43.386018 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:43.363549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44534 (local address 127.8.208.126:37897)
0929 03:01:43.364331 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:43.364342 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:43.364381 (+ 39us) server_negotiation.cc:407] Connection header received
0929 03:01:43.366029 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:43.366049 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:43.366430 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:43.366735 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:43.368480 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:43.369810 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:43.371672 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:43.372319 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:43.372440 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:43.377477 (+ 5037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:43.377490 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:43.377509 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:43.377566 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:43.381637 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:43.382296 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:43.382310 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:43.382320 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:43.382532 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:43.382906 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:43.382914 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:43.382918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:43.383568 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:43.383868 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:43.384185 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 03:01:43.384659 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":392,"threads_started":1}
I20260929 03:01:43.388924 24567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } }
I20260929 03:01:43.389297 24567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:43.389379 24567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76455a2e30954eb5a0e057e624da58c9, State: Initialized, Role: FOLLOWER
I20260929 03:01:43.389895 24567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } }
I20260929 03:01:43.390105 24567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:01:43.390200 24567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:01:43.390313 24567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:43.392295 24567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } }
I20260929 03:01:43.392699 24567 leader_election.cc:302] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76455a2e30954eb5a0e057e624da58c9; no voters:
I20260929 03:01:43.393208 24567 leader_election.cc:288] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:01:43.393352 24572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:01:43.394425 24572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 1 LEADER]: Becoming Leader. State: Replica: 76455a2e30954eb5a0e057e624da58c9, State: Running, Role: LEADER
I20260929 03:01:43.395020 24572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } }
I20260929 03:01:43.395448 24567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:01:43.397874 24573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76455a2e30954eb5a0e057e624da58c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } } }
I20260929 03:01:43.398370 24573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:43.400251 24574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76455a2e30954eb5a0e057e624da58c9. Latest consensus state: current_term: 1 leader_uuid: "76455a2e30954eb5a0e057e624da58c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } } }
I20260929 03:01:43.400646 24574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: This master's current role is: LEADER
I20260929 03:01:43.402671 24580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:01:43.406651 24580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:01:43.414309 24580 catalog_manager.cc:1347] Generated new cluster ID: d58f31b4d917450cb3d3e69bb0050c01
I20260929 03:01:43.414415 24580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:01:43.436012 24580 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:01:43.437953 24580 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:01:43.455159 24580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Generated new TSK 0
I20260929 03:01:43.456228 24580 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 03:01:43.514843 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37897
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:43.840909 24593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:43.841281 24593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:43.841388 24593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:43.848460 24593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:43.848577 24593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:43.848699 24593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:01:43.858462 24593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:43.859606 24593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:43.860824 24593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:43.874955 24599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:43.875083 24598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:43.875969 24601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:43.877825 24593 server_base.cc:1034] running on GCE node
I20260929 03:01:43.878567 24593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:43.879844 24593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:43.881122 24593 hybrid_clock.cc:648] HybridClock initialized: now 1790650903881012 us; error 83 us; skew 500 ppm
I20260929 03:01:43.881528 24593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:43 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650903, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:43.884642 24593 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 03:01:43.886142 24593 webserver.cc:460] Webserver started at http://127.8.208.65:44589/ using document root <none> and password file <none>
I20260929 03:01:43.886837 24593 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:43.886981 24593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:43.887315 24593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:01:43.890430 24593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4"
format_stamp: "Formatted at 2026-09-29 03:01:43 on dist-test-slave-m1rv"
server_key: "f49af919cb0c868fd412e9b32f117cdd"
server_key_iv: "76274b4cb60b1a98852840253563c78e"
server_key_version: "encryptionkey@0"
I20260929 03:01:43.891224 24593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4"
format_stamp: "Formatted at 2026-09-29 03:01:43 on dist-test-slave-m1rv"
server_key: "f49af919cb0c868fd412e9b32f117cdd"
server_key_iv: "76274b4cb60b1a98852840253563c78e"
server_key_version: "encryptionkey@0"
I20260929 03:01:43.896282 24593 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260929 03:01:43.899763 24608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:43.901129 24593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 03:01:43.901346 24593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4"
format_stamp: "Formatted at 2026-09-29 03:01:43 on dist-test-slave-m1rv"
server_key: "f49af919cb0c868fd412e9b32f117cdd"
server_key_iv: "76274b4cb60b1a98852840253563c78e"
server_key_version: "encryptionkey@0"
I20260929 03:01:43.901549 24593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:43.915352 24593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:43.919126 24593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:43.919415 24593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:43.920346 24593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:43.921661 24593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:43.921772 24593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:43.921916 24593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:43.921993 24593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:43.955454 24593 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:44533
I20260929 03:01:43.955541 24720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:44533 every 8 connection(s)
I20260929 03:01:43.956750 24593 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:01:43.966737 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 24593
I20260929 03:01:43.966938 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 03:01:43.967620 9025 external_mini_cluster.cc:1428] Setting key deb0d333e126aca5fe38c399053b56f7
Sep 29 03:01:43 dist-test-slave-m1rv krb5kdc[24487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650903, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:43.976421 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:43.959362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:49251 (local address 127.8.208.126:37897)
0929 03:01:43.959784 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:43.959789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:43.960736 (+ 947us) server_negotiation.cc:407] Connection header received
0929 03:01:43.962550 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:43.962566 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:43.962633 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:43.962732 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:43.964112 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:43.964846 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:43.966399 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:43.966790 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:43.966950 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:43.970657 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:43.970662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:43.970665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:43.970696 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:43.973390 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:43.974129 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:43.974133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:43.974136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:43.974251 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:43.974694 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:43.974699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:43.974702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:43.974962 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:43.975132 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:43.975820 (+ 688us) server_negotiation.cc:299] Negotiation successful
0929 03:01:43.976064 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":206,"threads_started":1}
I20260929 03:01:43.977439 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:43.959923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:49251)
0929 03:01:43.960582 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:43.960623 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:43.962276 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:43.962925 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:43.962932 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:43.963413 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:43.963956 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:43.963972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:43.964973 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:43.964977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:43.966236 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:43.966247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:43.967237 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:43.967242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:43.967437 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:43.968134 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:43.968156 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:43.970425 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:43.973538 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:43.973547 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:43.973572 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:43.973982 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:43.974362 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:43.974368 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:43.974371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:43.974572 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:43.975094 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:43.975102 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:43.975441 (+ 339us) client_negotiation.cc:769] Sending connection context
0929 03:01:43.975706 (+ 265us) client_negotiation.cc:240] Negotiation successful
0929 03:01:43.976225 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":251,"threads_started":1}
I20260929 03:01:43.979007 24721 heartbeater.cc:347] Connected to a master server at 127.8.208.126:37897
I20260929 03:01:43.979336 24721 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:43.980185 24721 heartbeater.cc:511] Master 127.8.208.126:37897 requested a full tablet report, sending...
I20260929 03:01:43.982527 24532 ts_manager.cc:194] Registered new tserver with Master: 29bb1d4dfb3a4394ad622ffdb6a68ab4 (127.8.208.65:44533)
I20260929 03:01:43.984633 24532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:49251
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 03:01:44.022167 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37897
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:44.339443 24727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:44.339720 24727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:44.339814 24727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:44.346786 24727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:44.346887 24727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:44.346997 24727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:01:44.356235 24727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:44.357441 24727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:44.358692 24727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:44.371734 24735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:44.371675 24733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:44.372299 24732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:44.372488 24727 server_base.cc:1034] running on GCE node
I20260929 03:01:44.373080 24727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:44.374003 24727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:44.375231 24727 hybrid_clock.cc:648] HybridClock initialized: now 1790650904375188 us; error 58 us; skew 500 ppm
I20260929 03:01:44.375523 24727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:44 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650904, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:44.378556 24727 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 03:01:44.379804 24727 webserver.cc:460] Webserver started at http://127.8.208.66:41895/ using document root <none> and password file <none>
I20260929 03:01:44.380396 24727 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:44.380489 24727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:44.380728 24727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:01:44.384150 24727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "0ce6f1338a894090986afc2f4a6e2bcb"
format_stamp: "Formatted at 2026-09-29 03:01:44 on dist-test-slave-m1rv"
server_key: "0dc7ef68625c8f435fbac0b944c03f89"
server_key_iv: "a19f197e47bc37ff66201f68186bc9dd"
server_key_version: "encryptionkey@0"
I20260929 03:01:44.384791 24727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "0ce6f1338a894090986afc2f4a6e2bcb"
format_stamp: "Formatted at 2026-09-29 03:01:44 on dist-test-slave-m1rv"
server_key: "0dc7ef68625c8f435fbac0b944c03f89"
server_key_iv: "a19f197e47bc37ff66201f68186bc9dd"
server_key_version: "encryptionkey@0"
I20260929 03:01:44.389509 24727 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260929 03:01:44.392576 24742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:44.393879 24727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:01:44.394003 24727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "0ce6f1338a894090986afc2f4a6e2bcb"
format_stamp: "Formatted at 2026-09-29 03:01:44 on dist-test-slave-m1rv"
server_key: "0dc7ef68625c8f435fbac0b944c03f89"
server_key_iv: "a19f197e47bc37ff66201f68186bc9dd"
server_key_version: "encryptionkey@0"
I20260929 03:01:44.394171 24727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:44.420521 24727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:44.424439 24727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:44.424682 24727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:44.425621 24727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:44.426863 24727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:44.426934 24727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:44.427023 24727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:44.427069 24727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:44.460239 24727 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:45305
I20260929 03:01:44.460296 24854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:45305 every 8 connection(s)
I20260929 03:01:44.461594 24727 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:01:44.463881 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 24727
I20260929 03:01:44.464079 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 03:01:44.464659 9025 external_mini_cluster.cc:1428] Setting key 27edc5424876a5697590ea936eea15a3
Sep 29 03:01:44 dist-test-slave-m1rv krb5kdc[24487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650904, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:44.484192 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:44.464316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:49769 (local address 127.8.208.126:37897)
0929 03:01:44.464547 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:44.464554 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:44.466725 (+ 2171us) server_negotiation.cc:407] Connection header received
0929 03:01:44.468874 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:44.468882 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:44.468973 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:44.469086 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:44.470674 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:44.471857 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:44.473605 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:44.474131 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:44.474227 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:44.478484 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:44.478488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:44.478492 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:44.478525 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:44.481104 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:44.481878 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:44.481888 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:44.481890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:44.482013 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:44.482460 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:44.482467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:44.482474 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:44.482741 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:44.482915 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:44.483600 (+ 685us) server_negotiation.cc:299] Negotiation successful
0929 03:01:44.483840 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 03:01:44.485262 24857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:44.464838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:49769)
0929 03:01:44.466562 (+ 1724us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:44.466611 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:44.468488 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:44.469398 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:44.469405 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:44.469924 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:44.470485 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:44.470503 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:44.472005 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:44.472008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:44.473460 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:44.473477 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:44.474368 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:44.474378 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:44.474732 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:44.475495 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:44.475529 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:44.478237 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:44.481272 (+ 3035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:44.481282 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:44.481300 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:44.481737 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:44.482132 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:44.482138 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:44.482141 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:44.482339 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:44.482871 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:44.482879 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:44.483224 (+ 345us) client_negotiation.cc:769] Sending connection context
0929 03:01:44.483492 (+ 268us) client_negotiation.cc:240] Negotiation successful
0929 03:01:44.484054 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1435,"thread_start_us":1215,"threads_started":1}
I20260929 03:01:44.486766 24855 heartbeater.cc:347] Connected to a master server at 127.8.208.126:37897
I20260929 03:01:44.487133 24855 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:44.488050 24855 heartbeater.cc:511] Master 127.8.208.126:37897 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
I20260929 03:01:44.489774 24532 ts_manager.cc:194] Registered new tserver with Master: 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305)
I20260929 03:01:44.490623 24532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:49769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 03:01:44.519734 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37897
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:01:44.840651 24860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:01:44.840909 24860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:01:44.840999 24860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:01:44.847916 24860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:01:44.848011 24860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:01:44.848124 24860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:01:44.857405 24860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:37897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:01:44.858538 24860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:01:44.859788 24860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:01:44.872603 24868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:44.872640 24866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:01:44.872675 24865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:01:44.873407 24860 server_base.cc:1034] running on GCE node
I20260929 03:01:44.874029 24860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:01:44.874825 24860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:01:44.876030 24860 hybrid_clock.cc:648] HybridClock initialized: now 1790650904875996 us; error 24 us; skew 500 ppm
I20260929 03:01:44.876283 24860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:01:44 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650904, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:01:44.879295 24860 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 03:01:44.880702 24860 webserver.cc:460] Webserver started at http://127.8.208.67:45479/ using document root <none> and password file <none>
I20260929 03:01:44.881459 24860 fs_manager.cc:359] Metadata directory not provided
I20260929 03:01:44.881608 24860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:01:44.881922 24860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:01:44.884644 24860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "3aa761b2b7724e83b3185b3343864296"
format_stamp: "Formatted at 2026-09-29 03:01:44 on dist-test-slave-m1rv"
server_key: "446c26daa59bc3ef0b97154855a829a5"
server_key_iv: "90a19db8199d990fdb869ff722c81583"
server_key_version: "encryptionkey@0"
I20260929 03:01:44.885443 24860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "3aa761b2b7724e83b3185b3343864296"
format_stamp: "Formatted at 2026-09-29 03:01:44 on dist-test-slave-m1rv"
server_key: "446c26daa59bc3ef0b97154855a829a5"
server_key_iv: "90a19db8199d990fdb869ff722c81583"
server_key_version: "encryptionkey@0"
I20260929 03:01:44.890281 24860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:01:44.893465 24875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:44.894651 24860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 03:01:44.894817 24860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "3aa761b2b7724e83b3185b3343864296"
format_stamp: "Formatted at 2026-09-29 03:01:44 on dist-test-slave-m1rv"
server_key: "446c26daa59bc3ef0b97154855a829a5"
server_key_iv: "90a19db8199d990fdb869ff722c81583"
server_key_version: "encryptionkey@0"
I20260929 03:01:44.894996 24860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:01:44.917636 24860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:01:44.921380 24860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:01:44.921653 24860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:01:44.922583 24860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:01:44.923803 24860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:01:44.923907 24860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:44.924063 24860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:01:44.924136 24860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:01:44.957156 24860 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:35431
I20260929 03:01:44.957213 24987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:35431 every 8 connection(s)
I20260929 03:01:44.958451 24860 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:01:44.961619 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 24860
I20260929 03:01:44.961825 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 03:01:44.962401 9025 external_mini_cluster.cc:1428] Setting key 6e460cf08fb1e9c521bd3f627f82038f
Sep 29 03:01:44 dist-test-slave-m1rv krb5kdc[24487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650904, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:01:44.978922 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:44.961274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:33813 (local address 127.8.208.126:37897)
0929 03:01:44.961500 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:44.961506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:44.962841 (+ 1335us) server_negotiation.cc:407] Connection header received
0929 03:01:44.965247 (+ 2406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:44.965254 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:44.965336 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:44.965460 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:44.967089 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:44.967831 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:44.969381 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:44.969787 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:44.969886 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:44.973206 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:44.973213 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:44.973217 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:44.973255 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:44.975647 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:44.976521 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:44.976528 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:44.976532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:44.976736 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:44.977219 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:44.977224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:44.977228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:44.977514 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:44.977715 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:44.978356 (+ 641us) server_negotiation.cc:299] Negotiation successful
0929 03:01:44.978585 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 03:01:44.979844 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:44.961793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:33813)
0929 03:01:44.962637 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:44.962699 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:44.964955 (+ 2256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:44.965759 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:44.965768 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:44.966262 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:01:44.966937 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:44.966960 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:44.967950 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:44.967953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:44.969154 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:44.969166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:44.969964 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:44.969969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:44.970171 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:44.970861 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:01:44.970882 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:01:44.972973 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:01:44.975792 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:44.975800 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:44.975815 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:44.976396 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:44.976869 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:01:44.976874 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:01:44.976877 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:01:44.977060 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:01:44.977665 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:01:44.977672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:01:44.977995 (+ 323us) client_negotiation.cc:769] Sending connection context
0929 03:01:44.978258 (+ 263us) client_negotiation.cc:240] Negotiation successful
0929 03:01:44.978742 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":281,"threads_started":1}
I20260929 03:01:44.981122 24988 heartbeater.cc:347] Connected to a master server at 127.8.208.126:37897
I20260929 03:01:44.981420 24988 heartbeater.cc:464] Registering TS with master...
I20260929 03:01:44.982048 24988 heartbeater.cc:511] Master 127.8.208.126:37897 requested a full tablet report, sending...
I20260929 03:01:44.983341 24532 ts_manager.cc:194] Registered new tserver with Master: 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431)
I20260929 03:01:44.984122 24532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:33813
I20260929 03:01:44.988350 24721 heartbeater.cc:503] Master 127.8.208.126:37897 was elected leader, sending a full tablet report...
I20260929 03:01:44.990137 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:01:45.007352 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:44.994464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44538 (local address 127.8.208.126:37897)
0929 03:01:44.994612 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:44.994616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:44.995053 (+ 437us) server_negotiation.cc:407] Connection header received
0929 03:01:44.995290 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:44.995295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:44.995362 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:44.995462 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:44.997005 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:44.997837 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:44.999383 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:44.999795 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:44.999945 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.002423 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:45.002427 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:45.002431 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:45.002460 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:45.004869 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:45.005628 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:45.005631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:45.005634 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:45.005757 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:45.006111 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:45.006114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:45.006116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:45.006409 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:45.006597 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:45.006872 (+ 275us) server_negotiation.cc:299] Negotiation successful
0929 03:01:45.007041 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 03:01:45.011466 24531 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:01:45.018422 24531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:01:45.042027 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.029798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44533 (local address 127.0.0.1:32986)
0929 03:01:45.030659 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:45.030715 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:45.030918 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:45.032230 (+ 1312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:45.032241 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:45.032282 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:45.032736 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:45.032752 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.035340 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.035349 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:45.038355 (+ 3006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:45.038376 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.040750 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.040759 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:45.041034 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.041124 (+ 90us) client_negotiation.cc:769] Sending connection context
0929 03:01:45.041274 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 03:01:45.041525 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":367,"threads_started":1}
I20260929 03:01:45.043198 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.030027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32986 (local address 127.8.208.65:44533)
0929 03:01:45.031566 (+ 1539us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:45.031575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:45.031604 (+ 29us) server_negotiation.cc:407] Connection header received
0929 03:01:45.031711 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:45.031722 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:45.031961 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:45.032983 (+ 1022us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:45.033287 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.035159 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.038605 (+ 3446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.040475 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.042349 (+ 1874us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.042608 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:45.042690 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 03:01:45.042840 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1347,"thread_start_us":4107,"threads_started":1}
I20260929 03:01:45.047247 24656 tablet_service.cc:1467] Processing CreateTablet for tablet ae2c16eb811f4de6994ce4882f81f8e7 (DEFAULT_TABLE table=test-table [id=df627a906c394a5a97af9f1e25f0922a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:01:45.048756 24656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae2c16eb811f4de6994ce4882f81f8e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:45.050446 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.030910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35431 (local address 127.0.0.1:46032)
0929 03:01:45.042332 (+ 11422us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:45.042349 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:45.042457 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:45.043172 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:45.043179 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:45.043205 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:45.043599 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:45.043630 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.045960 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.045966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:45.047900 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:45.047938 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.049582 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.049588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:45.049770 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.049803 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 03:01:45.049906 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 03:01:45.050036 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11301,"mutex_wait_us":63,"thread_start_us":11169,"threads_started":1}
I20260929 03:01:45.050499 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.031512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46032 (local address 127.8.208.67:35431)
0929 03:01:45.041858 (+ 10346us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:45.041866 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:45.042600 (+ 734us) server_negotiation.cc:407] Connection header received
0929 03:01:45.042716 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:45.042723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:45.042967 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:45.043859 (+ 892us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:45.044124 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.045820 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.048098 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.049371 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.049506 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.049751 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:45.049959 (+ 208us) server_negotiation.cc:299] Negotiation successful
0929 03:01:45.050156 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10147,"thread_start_us":10181,"threads_started":1}
I20260929 03:01:45.050993 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.031357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:45305 (local address 127.0.0.1:36124)
0929 03:01:45.042646 (+ 11289us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:45.042680 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:45.042790 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:45.044166 (+ 1376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:45.044172 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:45.044202 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:45.044403 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:45.044413 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.046360 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.046370 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:45.048368 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:45.048384 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.050210 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.050216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:45.050408 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.050441 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 03:01:45.050509 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 03:01:45.050653 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11127,"mutex_wait_us":60,"thread_start_us":13029,"threads_started":1}
I20260929 03:01:45.051573 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.031895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36124 (local address 127.8.208.66:45305)
0929 03:01:45.043596 (+ 11701us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:45.043606 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:45.043635 (+ 29us) server_negotiation.cc:407] Connection header received
0929 03:01:45.043731 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:45.043737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:45.043959 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:45.044161 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:45.044608 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.046181 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.048544 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.050016 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.050812 (+ 796us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.050994 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:45.051075 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 03:01:45.051248 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11555,"thread_start_us":11406,"threads_started":1}
I20260929 03:01:45.053977 24923 tablet_service.cc:1467] Processing CreateTablet for tablet ae2c16eb811f4de6994ce4882f81f8e7 (DEFAULT_TABLE table=test-table [id=df627a906c394a5a97af9f1e25f0922a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:01:45.054385 24790 tablet_service.cc:1467] Processing CreateTablet for tablet ae2c16eb811f4de6994ce4882f81f8e7 (DEFAULT_TABLE table=test-table [id=df627a906c394a5a97af9f1e25f0922a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:01:45.055177 24923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae2c16eb811f4de6994ce4882f81f8e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:45.055608 24790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae2c16eb811f4de6994ce4882f81f8e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:01:45.065543 25006 tablet_bootstrap.cc:492] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296: Bootstrap starting.
I20260929 03:01:45.069062 25006 tablet_bootstrap.cc:654] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:45.070235 25006 log.cc:824] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:45.070803 25007 tablet_bootstrap.cc:492] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Bootstrap starting.
I20260929 03:01:45.072279 25009 tablet_bootstrap.cc:492] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb: Bootstrap starting.
I20260929 03:01:45.074003 25006 tablet_bootstrap.cc:492] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296: No bootstrap required, opened a new log
I20260929 03:01:45.074312 25006 ts_tablet_manager.cc:1397] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 03:01:45.074976 25007 tablet_bootstrap.cc:654] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:45.075721 25009 tablet_bootstrap.cc:654] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb: Neither blocks nor log segments found. Creating new log.
I20260929 03:01:45.076196 25007 log.cc:824] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:45.076960 25009 log.cc:824] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb: Log is configured to *not* fsync() on all Append() calls
I20260929 03:01:45.081992 25006 raft_consensus.cc:348] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } }
I20260929 03:01:45.082433 25006 raft_consensus.cc:374] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:45.082537 25006 raft_consensus.cc:729] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa761b2b7724e83b3185b3343864296, State: Initialized, Role: FOLLOWER
I20260929 03:01:45.083339 25009 tablet_bootstrap.cc:492] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb: No bootstrap required, opened a new log
I20260929 03:01:45.083151 25006 consensus_queue.cc:260] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } }
I20260929 03:01:45.083653 25009 ts_tablet_manager.cc:1397] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260929 03:01:45.083683 25007 tablet_bootstrap.cc:492] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: No bootstrap required, opened a new log
I20260929 03:01:45.083938 25007 ts_tablet_manager.cc:1397] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.010s
I20260929 03:01:45.084249 24988 heartbeater.cc:503] Master 127.8.208.126:37897 was elected leader, sending a full tablet report...
I20260929 03:01:45.084895 25006 ts_tablet_manager.cc:1428] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260929 03:01:45.093282 25009 raft_consensus.cc:348] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } }
I20260929 03:01:45.093390 25007 raft_consensus.cc:348] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } }
I20260929 03:01:45.093679 25009 raft_consensus.cc:374] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:45.093743 25007 raft_consensus.cc:374] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:01:45.093782 25009 raft_consensus.cc:729] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ce6f1338a894090986afc2f4a6e2bcb, State: Initialized, Role: FOLLOWER
I20260929 03:01:45.093863 25007 raft_consensus.cc:729] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29bb1d4dfb3a4394ad622ffdb6a68ab4, State: Initialized, Role: FOLLOWER
I20260929 03:01:45.094410 25009 consensus_queue.cc:260] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } }
I20260929 03:01:45.094494 25007 consensus_queue.cc:260] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } }
I20260929 03:01:45.095526 24855 heartbeater.cc:503] Master 127.8.208.126:37897 was elected leader, sending a full tablet report...
I20260929 03:01:45.096272 25009 ts_tablet_manager.cc:1428] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260929 03:01:45.097371 25007 ts_tablet_manager.cc:1428] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260929 03:01:45.209349 24722 tablet.cc:2369] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:01:45.210239 24989 tablet.cc:2369] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:01:45.213460 24856 tablet.cc:2369] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:01:45.408716 25014 raft_consensus.cc:482] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:01:45.409062 25014 raft_consensus.cc:504] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } }
I20260929 03:01:45.411212 25014 leader_election.cc:288] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431), 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305)
I20260929 03:01:45.420536 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.412128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:53607 (local address 127.8.208.66:45305)
0929 03:01:45.412376 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:45.412382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:45.412743 (+ 361us) server_negotiation.cc:407] Connection header received
0929 03:01:45.412864 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:45.412868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:45.412926 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:45.413001 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:45.413856 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.414943 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.418013 (+ 3070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.419261 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.419424 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.419498 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:45.420129 (+ 631us) server_negotiation.cc:299] Negotiation successful
0929 03:01:45.420263 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 03:01:45.420533 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.411983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:56167 (local address 127.8.208.67:35431)
0929 03:01:45.412173 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:45.412179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:45.412299 (+ 120us) server_negotiation.cc:407] Connection header received
0929 03:01:45.412633 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:45.412640 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:45.412729 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:45.412793 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:01:45.413857 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.414943 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.418013 (+ 3070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.419276 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.419426 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.419498 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:45.420129 (+ 631us) server_negotiation.cc:299] Negotiation successful
0929 03:01:45.420263 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 03:01:45.420804 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.411504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35431 (local address 127.8.208.65:56167)
0929 03:01:45.412183 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:45.412204 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:45.412432 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:45.413016 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:45.413028 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:45.413052 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:45.413689 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:45.413701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.415092 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.415097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:45.417841 (+ 2744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:45.417856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.419559 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.419564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:45.419843 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.419970 (+ 127us) client_negotiation.cc:769] Sending connection context
0929 03:01:45.420069 (+ 99us) client_negotiation.cc:240] Negotiation successful
0929 03:01:45.420296 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":359,"thread_start_us":876,"threads_started":1}
I20260929 03:01:45.420830 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.411543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:45305 (local address 127.8.208.65:53607)
0929 03:01:45.412648 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0929 03:01:45.412664 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:01:45.412765 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:01:45.413030 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:01:45.413034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:01:45.413116 (+ 82us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:01:45.413688 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:45.413701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.415094 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.415098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:45.417842 (+ 2744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:01:45.417856 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.419553 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.419559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:01:45.419843 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.419972 (+ 129us) client_negotiation.cc:769] Sending connection context
0929 03:01:45.420069 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 03:01:45.420296 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":964,"mutex_wait_us":426,"spinlock_wait_cycles":21376,"thread_start_us":810,"threads_started":1}
I20260929 03:01:45.421844 24943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae2c16eb811f4de6994ce4882f81f8e7" candidate_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa761b2b7724e83b3185b3343864296" is_pre_election: true
I20260929 03:01:45.422209 24810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae2c16eb811f4de6994ce4882f81f8e7" candidate_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" is_pre_election: true
I20260929 03:01:45.422398 24943 raft_consensus.cc:2454] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29bb1d4dfb3a4394ad622ffdb6a68ab4 in term 0.
I20260929 03:01:45.422765 24810 raft_consensus.cc:2454] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29bb1d4dfb3a4394ad622ffdb6a68ab4 in term 0.
I20260929 03:01:45.423105 24610 leader_election.cc:302] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29bb1d4dfb3a4394ad622ffdb6a68ab4, 3aa761b2b7724e83b3185b3343864296; no voters:
I20260929 03:01:45.423532 25014 raft_consensus.cc:2781] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:01:45.423691 25014 raft_consensus.cc:482] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:01:45.423812 25014 raft_consensus.cc:3037] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:45.426162 25014 raft_consensus.cc:504] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } }
I20260929 03:01:45.426729 25014 leader_election.cc:288] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431), 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305)
I20260929 03:01:45.427237 24943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae2c16eb811f4de6994ce4882f81f8e7" candidate_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa761b2b7724e83b3185b3343864296"
I20260929 03:01:45.427237 24810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae2c16eb811f4de6994ce4882f81f8e7" candidate_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ce6f1338a894090986afc2f4a6e2bcb"
I20260929 03:01:45.427439 24810 raft_consensus.cc:3037] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:45.427439 24943 raft_consensus.cc:3037] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:01:45.430593 24943 raft_consensus.cc:2454] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29bb1d4dfb3a4394ad622ffdb6a68ab4 in term 1.
I20260929 03:01:45.430590 24810 raft_consensus.cc:2454] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29bb1d4dfb3a4394ad622ffdb6a68ab4 in term 1.
I20260929 03:01:45.430979 24610 leader_election.cc:302] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29bb1d4dfb3a4394ad622ffdb6a68ab4, 3aa761b2b7724e83b3185b3343864296; no voters:
I20260929 03:01:45.431253 25014 raft_consensus.cc:2781] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:01:45.431690 25014 raft_consensus.cc:686] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 LEADER]: Becoming Leader. State: Replica: 29bb1d4dfb3a4394ad622ffdb6a68ab4, State: Running, Role: LEADER
I20260929 03:01:45.432260 25014 consensus_queue.cc:237] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } }
I20260929 03:01:45.436651 24531 catalog_manager.cc:5495] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29bb1d4dfb3a4394ad622ffdb6a68ab4 (127.8.208.65). New cstate: current_term: 1 leader_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } health_report { overall_health: UNKNOWN } } }
I20260929 03:01:45.492661 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.487400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.8.208.65:44533)
0929 03:01:45.487623 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:45.487627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:45.487647 (+ 20us) server_negotiation.cc:407] Connection header received
0929 03:01:45.487799 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:45.487803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:45.487892 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:45.488013 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:01:45.488395 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.489107 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.490555 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.490931 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.491019 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.491485 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:01:45.492069 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:01:45.492177 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:45.492277 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 03:01:45.492340 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 03:01:45.501128 24943 raft_consensus.cc:1261] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296 [term 1 FOLLOWER]: Refusing update from remote peer 29bb1d4dfb3a4394ad622ffdb6a68ab4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:01:45.501137 24810 raft_consensus.cc:1261] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb [term 1 FOLLOWER]: Refusing update from remote peer 29bb1d4dfb3a4394ad622ffdb6a68ab4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:01:45.502159 25017 consensus_queue.cc:1035] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:01:45.502476 25014 consensus_queue.cc:1035] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:01:45.516235 25021 mvcc.cc:205] Tried to move back new op lower bound from 7334506108922732544 to 7334506108654817280. Current Snapshot: MvccSnapshot[applied={T|T < 7334506108922732544}]
I20260929 03:01:45.529434 24531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:01:45.529667 24531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:01:45.533051 24531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Sending DeleteTablet for 3 replicas of tablet ae2c16eb811f4de6994ce4882f81f8e7
I20260929 03:01:45.534121 24656 tablet_service.cc:1514] Processing DeleteTablet for tablet ae2c16eb811f4de6994ce4882f81f8e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:01:45 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:32986
I20260929 03:01:45.534282 24923 tablet_service.cc:1514] Processing DeleteTablet for tablet ae2c16eb811f4de6994ce4882f81f8e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:01:45 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:46032
I20260929 03:01:45.534334 24790 tablet_service.cc:1514] Processing DeleteTablet for tablet ae2c16eb811f4de6994ce4882f81f8e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:01:45 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:36124
I20260929 03:01:45.539470 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 24593
I20260929 03:01:45.546531 25030 tablet_replica.cc:330] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: stopping tablet replica
I20260929 03:01:45.546527 25031 tablet_replica.cc:330] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296: stopping tablet replica
I20260929 03:01:45.547037 25031 raft_consensus.cc:2229] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:01:45.547072 25030 raft_consensus.cc:2229] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:01:45.547597 25031 raft_consensus.cc:2258] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:01:45.547935 25030 raft_consensus.cc:2258] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:01:45.548007 25032 tablet_replica.cc:330] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb: stopping tablet replica
I20260929 03:01:45.548470 25032 raft_consensus.cc:2229] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:01:45.549259 25032 raft_consensus.cc:2258] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:01:45.549891 25031 ts_tablet_manager.cc:1905] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:01:45.549891 25030 ts_tablet_manager.cc:1905] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:01:45.551481 25032 ts_tablet_manager.cc:1905] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:01:45.556875 25031 ts_tablet_manager.cc:1918] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:01:45.557113 25031 log.cc:1196] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/ae2c16eb811f4de6994ce4882f81f8e7
I20260929 03:01:45.557425 25030 ts_tablet_manager.cc:1918] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:01:45.557667 25030 log.cc:1196] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/ae2c16eb811f4de6994ce4882f81f8e7
I20260929 03:01:45.557827 25031 ts_tablet_manager.cc:1939] T ae2c16eb811f4de6994ce4882f81f8e7 P 3aa761b2b7724e83b3185b3343864296: Deleting consensus metadata
I20260929 03:01:45.558303 25030 ts_tablet_manager.cc:1939] T ae2c16eb811f4de6994ce4882f81f8e7 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Deleting consensus metadata
I20260929 03:01:45.558396 25032 ts_tablet_manager.cc:1918] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:01:45.559803 24517 catalog_manager.cc:4841] TS 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431): tablet ae2c16eb811f4de6994ce4882f81f8e7 (table test-table [id=df627a906c394a5a97af9f1e25f0922a]) successfully deleted
I20260929 03:01:45.560266 24517 catalog_manager.cc:4841] TS 29bb1d4dfb3a4394ad622ffdb6a68ab4 (127.8.208.65:44533): tablet ae2c16eb811f4de6994ce4882f81f8e7 (table test-table [id=df627a906c394a5a97af9f1e25f0922a]) successfully deleted
I20260929 03:01:45.560411 25032 log.cc:1196] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/ae2c16eb811f4de6994ce4882f81f8e7
I20260929 03:01:45.560881 25032 ts_tablet_manager.cc:1939] T ae2c16eb811f4de6994ce4882f81f8e7 P 0ce6f1338a894090986afc2f4a6e2bcb: Deleting consensus metadata
I20260929 03:01:45.562600 24516 catalog_manager.cc:4841] TS 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305): tablet ae2c16eb811f4de6994ce4882f81f8e7 (table test-table [id=df627a906c394a5a97af9f1e25f0922a]) successfully deleted
Sep 29 03:01:45 dist-test-slave-m1rv krb5kdc[24487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 03:01:45.576983 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.540898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.8.208.65:44533)
0929 03:01:45.541124 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:45.541132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:45.559000 (+ 17868us) server_negotiation.cc:407] Connection header received
0929 03:01:45.559304 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:45.559312 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:45.559430 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:45.559511 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:45.561623 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.562747 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.564798 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.565289 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.565386 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.570660 (+ 5274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:45.570672 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:45.570687 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:45.570727 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:45.573993 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:45.574831 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:45.574839 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:45.574841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:45.574980 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:45.575387 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:45.575391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:45.575393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:45.575735 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:45.576060 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:45.576369 (+ 309us) server_negotiation.cc:299] Negotiation successful
0929 03:01:45.576627 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 03:01:45.577742 24716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:45.676184 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 24593
I20260929 03:01:45.693970 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 24727
Sep 29 03:01:45 dist-test-slave-m1rv krb5kdc[24487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 03:01:45.709072 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.694755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36132 (local address 127.8.208.66:45305)
0929 03:01:45.694935 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:45.694939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:45.694957 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:01:45.695105 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:45.695109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:45.695169 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:45.695285 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:45.696627 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.697458 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.698902 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.699286 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.699379 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.703978 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:45.703992 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:45.704007 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:45.704051 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:45.706703 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:45.707367 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:45.707373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:45.707376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:45.707498 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:45.707833 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:45.707836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:45.707838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:45.708183 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:45.708394 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:45.708650 (+ 256us) server_negotiation.cc:299] Negotiation successful
0929 03:01:45.708798 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 03:01:45.709738 24850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:45.800340 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 24727
I20260929 03:01:45.817449 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 24860
Sep 29 03:01:45 dist-test-slave-m1rv krb5kdc[24487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 03:01:45.832317 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:01:45.818233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.8.208.67:35431)
0929 03:01:45.818425 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 03:01:45.818430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:01:45.818448 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:01:45.818646 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:01:45.818650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:01:45.818716 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:01:45.818805 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:01:45.820061 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.820762 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.822140 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:01:45.822561 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:01:45.822663 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:01:45.826976 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:01:45.826991 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:01:45.827007 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:01:45.827051 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:01:45.829928 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:45.830568 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:45.830574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:45.830577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:45.830710 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:01:45.831049 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:01:45.831052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:01:45.831054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:01:45.831391 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:01:45.831597 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0929 03:01:45.831880 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 03:01:45.832045 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 03:01:45.832966 24983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:45.928580 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 24860
I20260929 03:01:45.944782 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 24500
I20260929 03:01:45.946013 24562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:01:46.057067 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 24500
I20260929 03:02:06.079110 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37897
--webserver_interface=127.8.208.126
--webserver_port=38337
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:37897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:02:06.410821 25038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:06.411177 25038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:06.411343 25038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:06.418736 25038 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 03:02:06.418889 25038 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 03:02:06.418988 25038 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 03:02:06.419096 25038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:02:06.419198 25038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:06.419298 25038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:06.419387 25038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:02:06.419467 25038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:02:06.429972 25038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:37897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=38337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:06.431514 25038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:06.432926 25038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:06.457332 25044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:06.457281 25047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:06.457247 25045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:06.458272 25038 server_base.cc:1034] running on GCE node
I20260929 03:02:06.459255 25038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:06.460955 25038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:06.462211 25038 hybrid_clock.cc:648] HybridClock initialized: now 1790650926462115 us; error 72 us; skew 500 ppm
I20260929 03:02:06.462627 25038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:06 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650926, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:06.466588 25038 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:02:06.468354 25038 webserver.cc:460] Webserver started at http://127.8.208.126:38337/ using document root <none> and password file <none>
I20260929 03:02:06.469220 25038 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:06.469367 25038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:06.475816 25038 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:02:06.479183 25054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:06.480674 25038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:02:06.480893 25038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "76455a2e30954eb5a0e057e624da58c9"
format_stamp: "Formatted at 2026-09-29 03:01:43 on dist-test-slave-m1rv"
server_key: "139a53459a770f13f2455028582f387b"
server_key_iv: "e45ac597e02bbf88cf771345701294fa"
server_key_version: "encryptionkey@0"
I20260929 03:02:06.482128 25038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:06.510677 25038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:06.515319 25038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:06.515650 25038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:06.535362 25038 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:37897
I20260929 03:02:06.535392 25105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:37897 every 8 connection(s)
I20260929 03:02:06.537021 25038 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:02:06.543404 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 25038
I20260929 03:02:06.544292 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:44533
--local_ip_for_outbound_sockets=127.8.208.65
--tserver_master_addrs=127.8.208.126:37897
--webserver_port=44589
--webserver_interface=127.8.208.65
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:02:06.544550 25106 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:02:06.548126 25106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Bootstrap starting.
I20260929 03:02:06.568418 25106 log.cc:824] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:06.580368 25106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:02:06.580928 25106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Bootstrap complete.
I20260929 03:02:06.590356 25106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } }
I20260929 03:02:06.591305 25106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76455a2e30954eb5a0e057e624da58c9, State: Initialized, Role: FOLLOWER
I20260929 03:02:06.591950 25106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } }
I20260929 03:02:06.592206 25106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:02:06.592388 25106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:02:06.592564 25106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 1 FOLLOWER]: Advancing to term 2
I20260929 03:02:06.596547 25106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } }
I20260929 03:02:06.597098 25106 leader_election.cc:302] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76455a2e30954eb5a0e057e624da58c9; no voters:
I20260929 03:02:06.597800 25106 leader_election.cc:288] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 03:02:06.597941 25110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 03:02:06.599237 25110 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 2 LEADER]: Becoming Leader. State: Replica: 76455a2e30954eb5a0e057e624da58c9, State: Running, Role: LEADER
I20260929 03:02:06.599854 25110 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } }
I20260929 03:02:06.600252 25106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:02:06.602553 25112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76455a2e30954eb5a0e057e624da58c9. Latest consensus state: current_term: 2 leader_uuid: "76455a2e30954eb5a0e057e624da58c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } } }
I20260929 03:02:06.602916 25112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: This master's current role is: LEADER
I20260929 03:02:06.603466 25111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "76455a2e30954eb5a0e057e624da58c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } } }
I20260929 03:02:06.603768 25111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: This master's current role is: LEADER
I20260929 03:02:06.613216 25117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:02:06.621790 25117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:02:06.623260 25117 catalog_manager.cc:1259] Loaded cluster ID: d58f31b4d917450cb3d3e69bb0050c01
I20260929 03:02:06.623415 25117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:02:06.627482 25117 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:02:06.629256 25117 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Loaded TSK: 0
I20260929 03:02:06.659157 25117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Generated new TSK 1
I20260929 03:02:06.659554 25117 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 03:02:06.897353 25108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:06.897632 25108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:06.897789 25108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:06.905303 25108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:06.905412 25108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:06.905577 25108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:02:06.915535 25108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:44533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=44589
--tserver_master_addrs=127.8.208.126:37897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:06.916848 25108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:06.918244 25108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:06.933395 25133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:06.933434 25134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:06.933606 25136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:06.938263 25108 server_base.cc:1034] running on GCE node
I20260929 03:02:06.938918 25108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:06.939934 25108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:06.941242 25108 hybrid_clock.cc:648] HybridClock initialized: now 1790650926941116 us; error 55 us; skew 500 ppm
I20260929 03:02:06.941582 25108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:06 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650926, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:06.944751 25108 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 03:02:06.946475 25108 webserver.cc:460] Webserver started at http://127.8.208.65:44589/ using document root <none> and password file <none>
I20260929 03:02:06.947259 25108 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:06.947402 25108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:06.952843 25108 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:02:06.955763 25143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:06.957317 25108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 03:02:06.957485 25108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4"
format_stamp: "Formatted at 2026-09-29 03:01:43 on dist-test-slave-m1rv"
server_key: "f49af919cb0c868fd412e9b32f117cdd"
server_key_iv: "76274b4cb60b1a98852840253563c78e"
server_key_version: "encryptionkey@0"
I20260929 03:02:06.958453 25108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:06.994501 25108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:06.998719 25108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:06.998994 25108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:06.999934 25108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:07.001523 25108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:07.001631 25108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:07.001755 25108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:07.001857 25108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:07.036043 25108 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:44533
I20260929 03:02:07.036087 25255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:44533 every 8 connection(s)
I20260929 03:02:07.037634 25108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:02:07.038859 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 25108
I20260929 03:02:07.039638 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:45305
--local_ip_for_outbound_sockets=127.8.208.66
--tserver_master_addrs=127.8.208.126:37897
--webserver_port=41895
--webserver_interface=127.8.208.66
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:02:07 dist-test-slave-m1rv krb5kdc[24487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650926, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:02:07.073117 25259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:07.041324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:35491)
0929 03:02:07.049761 (+ 8437us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:07.049809 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:07.052707 (+ 2898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:07.054302 (+ 1595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:07.054316 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:07.055127 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:07.055887 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:07.055913 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.057545 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.057551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:07.059726 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:07.059744 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.060813 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.060822 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:07.061153 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:07.062120 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:02:07.062158 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:07.064942 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:07.068504 (+ 3562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:07.068515 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:07.068530 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:07.068957 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:07.069488 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:07.069493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:07.069496 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:07.069682 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:07.070634 (+ 952us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:07.070642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:07.070978 (+ 336us) client_negotiation.cc:769] Sending connection context
0929 03:02:07.071251 (+ 273us) client_negotiation.cc:240] Negotiation successful
0929 03:02:07.071797 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8169,"thread_start_us":7981,"threads_started":1}
I20260929 03:02:07.073114 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:07.041048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:35491 (local address 127.8.208.126:37897)
0929 03:02:07.041741 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:07.041752 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:07.049967 (+ 8215us) server_negotiation.cc:407] Connection header received
0929 03:02:07.053386 (+ 3419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:07.053400 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:07.053782 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:07.054096 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:07.056073 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.057375 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.059940 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.060579 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.060710 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:07.065295 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:07.065303 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:07.065318 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:07.065359 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:07.068344 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:07.069127 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:07.069133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:07.069139 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:07.069364 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:07.069793 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:07.069800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:07.069804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:07.070487 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:07.070693 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:07.071459 (+ 766us) server_negotiation.cc:299] Negotiation successful
0929 03:02:07.071843 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":337,"threads_started":1}
I20260929 03:02:07.076366 25256 heartbeater.cc:347] Connected to a master server at 127.8.208.126:37897
I20260929 03:02:07.076701 25256 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:07.077526 25256 heartbeater.cc:511] Master 127.8.208.126:37897 requested a full tablet report, sending...
I20260929 03:02:07.079909 25071 ts_manager.cc:194] Registered new tserver with Master: 29bb1d4dfb3a4394ad622ffdb6a68ab4 (127.8.208.65:44533)
I20260929 03:02:07.081808 25071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:35491
W20260929 03:02:07.377336 25260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:07.377602 25260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:07.377694 25260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:07.384610 25260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:07.384711 25260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:07.384817 25260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:02:07.395696 25260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:45305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=41895
--tserver_master_addrs=127.8.208.126:37897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:07.396925 25260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:07.398281 25260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:07.411412 25265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:07.411509 25266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:07.411446 25268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:07.412142 25260 server_base.cc:1034] running on GCE node
I20260929 03:02:07.412819 25260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:07.413826 25260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:07.415060 25260 hybrid_clock.cc:648] HybridClock initialized: now 1790650927415015 us; error 57 us; skew 500 ppm
I20260929 03:02:07.415377 25260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:07 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650927, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:07.418421 25260 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 03:02:07.419872 25260 webserver.cc:460] Webserver started at http://127.8.208.66:41895/ using document root <none> and password file <none>
I20260929 03:02:07.420547 25260 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:07.420706 25260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:07.425896 25260 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:02:07.428630 25275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:07.429991 25260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:02:07.430186 25260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "0ce6f1338a894090986afc2f4a6e2bcb"
format_stamp: "Formatted at 2026-09-29 03:01:44 on dist-test-slave-m1rv"
server_key: "0dc7ef68625c8f435fbac0b944c03f89"
server_key_iv: "a19f197e47bc37ff66201f68186bc9dd"
server_key_version: "encryptionkey@0"
I20260929 03:02:07.431123 25260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:07.460220 25260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:07.464277 25260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:07.464548 25260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:07.465544 25260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:07.466809 25260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:07.466933 25260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:07.467060 25260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:07.467147 25260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:07.499926 25260 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:45305
I20260929 03:02:07.500057 25387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:45305 every 8 connection(s)
I20260929 03:02:07.501430 25260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:02:07.509902 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 25260
I20260929 03:02:07.510708 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:35431
--local_ip_for_outbound_sockets=127.8.208.67
--tserver_master_addrs=127.8.208.126:37897
--webserver_port=45479
--webserver_interface=127.8.208.67
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:02:07 dist-test-slave-m1rv krb5kdc[24487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650927, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:02:07.531407 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:07.504006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:49479 (local address 127.8.208.126:37897)
0929 03:02:07.504198 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:07.504203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:07.505386 (+ 1183us) server_negotiation.cc:407] Connection header received
0929 03:02:07.507242 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:07.507246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:07.507314 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:07.507398 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:07.508789 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.509552 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.511989 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.512411 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.512509 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:07.525140 (+ 12631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:07.525146 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:07.525150 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:07.525233 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:07.528015 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:07.528772 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:07.528778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:07.528781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:07.528946 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:07.529446 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:07.529453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:07.529456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:07.529854 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:07.530020 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:07.530822 (+ 802us) server_negotiation.cc:299] Negotiation successful
0929 03:02:07.531068 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 03:02:07.532404 25390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:07.504511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:49479)
0929 03:02:07.505231 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:07.505269 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:07.506999 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:07.507611 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:07.507618 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:07.508101 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:07.508638 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:07.508667 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.509713 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.509720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:07.511834 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:07.511864 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.512645 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.512654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:07.512969 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:07.522486 (+ 9517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:02:07.522523 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:07.524838 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:07.528168 (+ 3330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:07.528178 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:07.528197 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:07.528641 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:07.529058 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:07.529063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:07.529067 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:07.529330 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:07.529979 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:07.529987 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:07.530455 (+ 468us) client_negotiation.cc:769] Sending connection context
0929 03:02:07.530730 (+ 275us) client_negotiation.cc:240] Negotiation successful
0929 03:02:07.531274 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":251,"threads_started":1}
I20260929 03:02:07.534209 25388 heartbeater.cc:347] Connected to a master server at 127.8.208.126:37897
I20260929 03:02:07.534631 25388 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:07.535648 25388 heartbeater.cc:511] Master 127.8.208.126:37897 requested a full tablet report, sending...
I20260929 03:02:07.537132 25071 ts_manager.cc:194] Registered new tserver with Master: 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305)
I20260929 03:02:07.537990 25071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:49479
W20260929 03:02:07.837153 25391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:07.837445 25391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:07.837621 25391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:07.844702 25391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:07.844807 25391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:07.844983 25391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:02:07.854379 25391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:35431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=45479
--tserver_master_addrs=127.8.208.126:37897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:07.855600 25391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:07.856940 25391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:07.869801 25396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:07.869890 25397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:07.869845 25399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:07.871021 25391 server_base.cc:1034] running on GCE node
I20260929 03:02:07.871810 25391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:07.873112 25391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:07.874377 25391 hybrid_clock.cc:648] HybridClock initialized: now 1790650927874301 us; error 54 us; skew 500 ppm
I20260929 03:02:07.874752 25391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:07 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650927, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:07.878613 25391 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 03:02:07.880033 25391 webserver.cc:460] Webserver started at http://127.8.208.67:45479/ using document root <none> and password file <none>
I20260929 03:02:07.880722 25391 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:07.880882 25391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:07.886202 25391 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 03:02:07.888736 25406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:07.890058 25391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:02:07.890236 25391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "3aa761b2b7724e83b3185b3343864296"
format_stamp: "Formatted at 2026-09-29 03:01:44 on dist-test-slave-m1rv"
server_key: "446c26daa59bc3ef0b97154855a829a5"
server_key_iv: "90a19db8199d990fdb869ff722c81583"
server_key_version: "encryptionkey@0"
I20260929 03:02:07.891197 25391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:07.918687 25391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:07.922686 25391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:07.922967 25391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:07.923936 25391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:07.925290 25391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:07.925421 25391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:07.925547 25391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:07.925645 25391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:07.958875 25391 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:35431
I20260929 03:02:07.958962 25518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:35431 every 8 connection(s)
I20260929 03:02:07.960301 25391 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:02:07.963661 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 25391
Sep 29 03:02:07 dist-test-slave-m1rv krb5kdc[24487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650927, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:02:07.980607 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:07.964572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33516 (local address 127.8.208.126:37897)
0929 03:02:07.965066 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:07.965072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:07.965635 (+ 563us) server_negotiation.cc:407] Connection header received
0929 03:02:07.965821 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:07.965825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:07.965890 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:07.965983 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:07.968099 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.968849 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.970530 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.970976 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.971072 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:07.974563 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:07.974568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:07.974571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:07.974600 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:07.977715 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:07.978551 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:07.978555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:07.978557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:07.978692 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:07.979109 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:07.979112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:07.979114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:07.979372 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:07.979582 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:07.979950 (+ 368us) server_negotiation.cc:299] Negotiation successful
0929 03:02:07.980164 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":253,"threads_started":1}
I20260929 03:02:07.983078 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:07.962803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:44025 (local address 127.8.208.126:37897)
0929 03:02:07.962975 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:07.962979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:07.964353 (+ 1374us) server_negotiation.cc:407] Connection header received
0929 03:02:07.967157 (+ 2804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:07.967160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:07.967212 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:07.967287 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:07.969018 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.969796 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.971815 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.972298 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.972389 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:07.976388 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:07.976393 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:07.976396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:07.976444 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:07.979546 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:07.980398 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:07.980402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:07.980406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:07.980514 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:07.981010 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:07.981015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:07.981018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:07.981412 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:07.981573 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:07.982480 (+ 907us) server_negotiation.cc:299] Negotiation successful
0929 03:02:07.982716 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 03:02:07.983919 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:07.963341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:44025)
0929 03:02:07.964174 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:07.964225 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:07.966839 (+ 2614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:07.967579 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:07.967592 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:07.968281 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:07.968830 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:07.968847 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.969964 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.969970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:07.971688 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:07.971700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.972523 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:07.972534 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:07.972837 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:07.973534 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:02:07.973568 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:07.976181 (+ 2613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:07.979682 (+ 3501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:07.979689 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:07.979701 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:07.980276 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:07.980635 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:07.980641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:07.980644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:07.980883 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:07.981560 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:07.981569 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:07.982033 (+ 464us) client_negotiation.cc:769] Sending connection context
0929 03:02:07.982361 (+ 328us) client_negotiation.cc:240] Negotiation successful
0929 03:02:07.982738 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":338,"threads_started":1}
I20260929 03:02:07.985252 25519 heartbeater.cc:347] Connected to a master server at 127.8.208.126:37897
I20260929 03:02:07.985555 25519 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:07.986310 25519 heartbeater.cc:511] Master 127.8.208.126:37897 requested a full tablet report, sending...
I20260929 03:02:07.987709 25071 ts_manager.cc:194] Registered new tserver with Master: 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431)
I20260929 03:02:07.988433 25071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:44025
I20260929 03:02:07.995801 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:02:08.002480 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:07.996788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33532 (local address 127.8.208.126:37897)
0929 03:02:07.996952 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:07.996956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:07.997336 (+ 380us) server_negotiation.cc:407] Connection header received
0929 03:02:07.997506 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:07.997510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:07.997634 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:07.997755 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:02:07.998237 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:07.999229 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.000790 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.001168 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.001347 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.001809 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:02:08.002013 (+ 204us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0929 03:02:08.002280 (+ 267us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33532: token expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260929 03:02:08.002617 25258 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33532: token expired
W20260929 03:02:08.003100 24994 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.208.126:37897)
I20260929 03:02:08.015141 25258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.003540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33542 (local address 127.8.208.126:37897)
0929 03:02:08.003687 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:08.003691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:08.003706 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:02:08.003805 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:08.003809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:08.003859 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:08.003934 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:08.005273 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.005969 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.007334 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.007697 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.007784 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.010255 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:08.010259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:08.010262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:08.010290 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:08.013039 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:08.013656 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:08.013660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:08.013662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:08.013768 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:08.014066 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:08.014069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:08.014072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:08.014339 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:08.014527 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:08.014749 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 03:02:08.014900 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:02:08.018386 25071 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:02:08.025843 25071 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:02:08.056097 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.038178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33842 (local address 127.8.208.67:35431)
0929 03:02:08.045376 (+ 7198us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:08.045385 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:08.047008 (+ 1623us) server_negotiation.cc:407] Connection header received
0929 03:02:08.047293 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:08.047301 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:08.047519 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:08.047677 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:08.048594 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.050208 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.053101 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.054641 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.054760 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.054985 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:08.055546 (+ 561us) server_negotiation.cc:299] Negotiation successful
0929 03:02:08.055720 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6976,"thread_start_us":6819,"threads_started":1}
I20260929 03:02:08.056212 25530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.037563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35431 (local address 127.0.0.1:33842)
0929 03:02:08.046912 (+ 9349us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.046931 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:08.047112 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:08.047987 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:08.047992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:08.048018 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:08.048351 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:08.048361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.050401 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.050409 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:08.052962 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:08.052972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.054928 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.054933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:08.055255 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.055334 (+ 79us) client_negotiation.cc:769] Sending connection context
0929 03:02:08.055485 (+ 151us) client_negotiation.cc:240] Negotiation successful
0929 03:02:08.055720 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9214,"mutex_wait_us":25,"thread_start_us":9053,"threads_started":1}
I20260929 03:02:08.056669 25529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.037486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:45305 (local address 127.0.0.1:39820)
0929 03:02:08.044099 (+ 6613us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.044146 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:08.044301 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:08.046247 (+ 1946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:08.046254 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:08.046292 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:08.046795 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:08.046810 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.049763 (+ 2953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.049768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:08.052515 (+ 2747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:08.052529 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.054873 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.054879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:08.055343 (+ 464us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.055373 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 03:02:08.055485 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 03:02:08.055720 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6379,"mutex_wait_us":251,"thread_start_us":6251,"threads_started":1}
I20260929 03:02:08.056846 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.037642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39820 (local address 127.8.208.66:45305)
0929 03:02:08.045756 (+ 8114us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:08.045763 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:08.045785 (+ 22us) server_negotiation.cc:407] Connection header received
0929 03:02:08.045860 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:08.045865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:08.046041 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:08.046240 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:08.047030 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.048593 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.052693 (+ 4100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.054404 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.054555 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.054772 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:08.056395 (+ 1623us) server_negotiation.cc:299] Negotiation successful
0929 03:02:08.056555 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7992,"thread_start_us":7910,"threads_started":1}
I20260929 03:02:08.061029 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.047322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57196 (local address 127.8.208.65:44533)
0929 03:02:08.051053 (+ 3731us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:08.051060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:08.054007 (+ 2947us) server_negotiation.cc:407] Connection header received
0929 03:02:08.054276 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:08.054284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:08.054492 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:08.054672 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:08.055657 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.057106 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.058967 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.059872 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.060023 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.060240 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:08.060498 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 03:02:08.060687 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3559,"thread_start_us":3408,"threads_started":1}
I20260929 03:02:08.060686 25323 tablet_service.cc:1467] Processing CreateTablet for tablet b0e9779f100b4722b7a5fb752f540422 (DEFAULT_TABLE table=test-table [id=ce7377b2a01341e08510a7c3f5739131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:02:08.060297 25454 tablet_service.cc:1467] Processing CreateTablet for tablet b0e9779f100b4722b7a5fb752f540422 (DEFAULT_TABLE table=test-table [id=ce7377b2a01341e08510a7c3f5739131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:02:08.061748 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.047000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44533 (local address 127.0.0.1:57196)
0929 03:02:08.053907 (+ 6907us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.053925 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:08.054104 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:08.054948 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:08.054954 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:08.054980 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:08.055440 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:08.055471 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.057321 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.057326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:08.058811 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:08.058822 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.060102 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.060108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:08.060287 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.060322 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 03:02:08.061273 (+ 951us) client_negotiation.cc:240] Negotiation successful
0929 03:02:08.061418 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6752,"mutex_wait_us":46,"thread_start_us":6618,"threads_started":1}
I20260929 03:02:08.062150 25323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e9779f100b4722b7a5fb752f540422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:08.062150 25454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e9779f100b4722b7a5fb752f540422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:08.068141 25191 tablet_service.cc:1467] Processing CreateTablet for tablet b0e9779f100b4722b7a5fb752f540422 (DEFAULT_TABLE table=test-table [id=ce7377b2a01341e08510a7c3f5739131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:02:08.069468 25191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e9779f100b4722b7a5fb752f540422. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:08.077839 25535 tablet_bootstrap.cc:492] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb: Bootstrap starting.
I20260929 03:02:08.078575 25536 tablet_bootstrap.cc:492] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296: Bootstrap starting.
I20260929 03:02:08.081468 25537 tablet_bootstrap.cc:492] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Bootstrap starting.
I20260929 03:02:08.082346 25536 tablet_bootstrap.cc:654] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:08.082386 25535 tablet_bootstrap.cc:654] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:08.083438 25535 log.cc:824] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:08.083590 25536 log.cc:824] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:08.084725 25537 tablet_bootstrap.cc:654] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:08.085284 25256 heartbeater.cc:503] Master 127.8.208.126:37897 was elected leader, sending a full tablet report...
I20260929 03:02:08.085980 25537 log.cc:824] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:08.086596 25535 tablet_bootstrap.cc:492] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb: No bootstrap required, opened a new log
I20260929 03:02:08.086874 25535 ts_tablet_manager.cc:1397] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 03:02:08.089571 25537 tablet_bootstrap.cc:492] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: No bootstrap required, opened a new log
I20260929 03:02:08.089862 25537 ts_tablet_manager.cc:1397] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260929 03:02:08.091338 25536 tablet_bootstrap.cc:492] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296: No bootstrap required, opened a new log
I20260929 03:02:08.091606 25536 ts_tablet_manager.cc:1397] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.003s
I20260929 03:02:08.094635 25535 raft_consensus.cc:348] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:08.095062 25535 raft_consensus.cc:374] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:08.095184 25535 raft_consensus.cc:729] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ce6f1338a894090986afc2f4a6e2bcb, State: Initialized, Role: FOLLOWER
I20260929 03:02:08.096400 25535 consensus_queue.cc:260] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:08.097574 25388 heartbeater.cc:503] Master 127.8.208.126:37897 was elected leader, sending a full tablet report...
I20260929 03:02:08.100471 25535 ts_tablet_manager.cc:1428] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb: Time spent starting tablet: real 0.013s user 0.006s sys 0.004s
I20260929 03:02:08.100543 25537 raft_consensus.cc:348] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:08.100958 25537 raft_consensus.cc:374] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:08.101061 25537 raft_consensus.cc:729] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29bb1d4dfb3a4394ad622ffdb6a68ab4, State: Initialized, Role: FOLLOWER
I20260929 03:02:08.101649 25537 consensus_queue.cc:260] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:08.102290 25536 raft_consensus.cc:348] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:08.102742 25536 raft_consensus.cc:374] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:08.102847 25536 raft_consensus.cc:729] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa761b2b7724e83b3185b3343864296, State: Initialized, Role: FOLLOWER
I20260929 03:02:08.103298 25537 ts_tablet_manager.cc:1428] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260929 03:02:08.103410 25536 consensus_queue.cc:260] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:08.104486 25519 heartbeater.cc:503] Master 127.8.208.126:37897 was elected leader, sending a full tablet report...
I20260929 03:02:08.105058 25536 ts_tablet_manager.cc:1428] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
W20260929 03:02:08.212033 25520 tablet.cc:2369] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:02:08.213726 25542 raft_consensus.cc:482] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:02:08.214023 25542 raft_consensus.cc:504] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:08.215667 25542 leader_election.cc:288] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305), 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431)
I20260929 03:02:08.222569 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.216186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:35355 (local address 127.8.208.66:45305)
0929 03:02:08.216409 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:08.216419 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:08.217218 (+ 799us) server_negotiation.cc:407] Connection header received
0929 03:02:08.217434 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:08.217438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:08.217521 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:08.217628 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:08.218241 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.218920 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.220824 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.221651 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.221825 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.221901 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:08.222184 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 03:02:08.222308 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 03:02:08.222764 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.216058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35431 (local address 127.8.208.65:35001)
0929 03:02:08.217009 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.217024 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:08.217160 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:08.217547 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:08.217552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:08.217580 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:08.217992 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:08.218002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.219029 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.219034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:08.220671 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:08.220682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.221923 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.221929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:08.222088 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.222109 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 03:02:08.222213 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 03:02:08.222338 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"mutex_wait_us":78,"thread_start_us":573,"threads_started":1}
I20260929 03:02:08.222635 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.216130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:35001 (local address 127.8.208.67:35431)
0929 03:02:08.216383 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:08.216392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:08.217131 (+ 739us) server_negotiation.cc:407] Connection header received
0929 03:02:08.217345 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:08.217352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:08.217438 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:08.217509 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:08.218151 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.218893 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.220876 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.221709 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.221889 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.221964 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:08.222240 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 03:02:08.222374 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 03:02:08.222813 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.216171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:45305 (local address 127.8.208.65:35355)
0929 03:02:08.217113 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.217131 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:08.217295 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:08.217666 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:08.217670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:08.217709 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:08.218054 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:08.218084 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.219076 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.219079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:08.220667 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:08.220678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.221869 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.221875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:08.222026 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.222049 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 03:02:08.222158 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 03:02:08.222282 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":20,"thread_start_us":523,"threads_started":1}
I20260929 03:02:08.223997 25343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e9779f100b4722b7a5fb752f540422" candidate_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" is_pre_election: true
I20260929 03:02:08.223999 25474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e9779f100b4722b7a5fb752f540422" candidate_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa761b2b7724e83b3185b3343864296" is_pre_election: true
I20260929 03:02:08.224584 25343 raft_consensus.cc:2454] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29bb1d4dfb3a4394ad622ffdb6a68ab4 in term 0.
I20260929 03:02:08.224577 25474 raft_consensus.cc:2454] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29bb1d4dfb3a4394ad622ffdb6a68ab4 in term 0.
I20260929 03:02:08.225306 25145 leader_election.cc:302] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29bb1d4dfb3a4394ad622ffdb6a68ab4, 3aa761b2b7724e83b3185b3343864296; no voters:
I20260929 03:02:08.225716 25542 raft_consensus.cc:2781] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:02:08.225898 25542 raft_consensus.cc:482] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:02:08.226001 25542 raft_consensus.cc:3037] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:08.228189 25542 raft_consensus.cc:504] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:08.228794 25542 leader_election.cc:288] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305), 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431)
I20260929 03:02:08.229311 25474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e9779f100b4722b7a5fb752f540422" candidate_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa761b2b7724e83b3185b3343864296"
I20260929 03:02:08.229311 25343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e9779f100b4722b7a5fb752f540422" candidate_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ce6f1338a894090986afc2f4a6e2bcb"
I20260929 03:02:08.229519 25343 raft_consensus.cc:3037] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:08.229538 25474 raft_consensus.cc:3037] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:08.232769 25474 raft_consensus.cc:2454] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29bb1d4dfb3a4394ad622ffdb6a68ab4 in term 1.
I20260929 03:02:08.232769 25343 raft_consensus.cc:2454] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29bb1d4dfb3a4394ad622ffdb6a68ab4 in term 1.
I20260929 03:02:08.233199 25145 leader_election.cc:302] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0ce6f1338a894090986afc2f4a6e2bcb, 29bb1d4dfb3a4394ad622ffdb6a68ab4, 3aa761b2b7724e83b3185b3343864296; no voters:
I20260929 03:02:08.233548 25542 raft_consensus.cc:2781] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:02:08.234062 25542 raft_consensus.cc:686] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 LEADER]: Becoming Leader. State: Replica: 29bb1d4dfb3a4394ad622ffdb6a68ab4, State: Running, Role: LEADER
I20260929 03:02:08.234567 25542 consensus_queue.cc:237] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:08.239038 25070 catalog_manager.cc:5495] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29bb1d4dfb3a4394ad622ffdb6a68ab4 (127.8.208.65). New cstate: current_term: 1 leader_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } health_report { overall_health: HEALTHY } } }
W20260929 03:02:08.253465 25389 tablet.cc:2369] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:02:08.289943 25257 tablet.cc:2369] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:02:08.293483 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.288112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57208 (local address 127.8.208.65:44533)
0929 03:02:08.288337 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:08.288343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:08.288368 (+ 25us) server_negotiation.cc:407] Connection header received
0929 03:02:08.288474 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:08.288478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:08.288560 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:08.288714 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:02:08.289074 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.289891 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.291385 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.291831 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.291965 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.292448 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:02:08.292967 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:02:08.293091 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:08.293212 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 03:02:08.293286 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 03:02:08.301924 25343 raft_consensus.cc:1261] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb [term 1 FOLLOWER]: Refusing update from remote peer 29bb1d4dfb3a4394ad622ffdb6a68ab4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:02:08.301924 25474 raft_consensus.cc:1261] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296 [term 1 FOLLOWER]: Refusing update from remote peer 29bb1d4dfb3a4394ad622ffdb6a68ab4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:02:08.303026 25546 consensus_queue.cc:1035] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:02:08.303413 25542 consensus_queue.cc:1035] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:02:08.332552 25071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:02:08.332846 25071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:02:08.336397 25071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Sending DeleteTablet for 3 replicas of tablet b0e9779f100b4722b7a5fb752f540422
I20260929 03:02:08.337600 25323 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e9779f100b4722b7a5fb752f540422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:02:08 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:39820
I20260929 03:02:08.337703 25454 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e9779f100b4722b7a5fb752f540422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:02:08 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:33842
I20260929 03:02:08.338070 25191 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e9779f100b4722b7a5fb752f540422 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:02:08 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:57196
I20260929 03:02:08.338887 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 25108
I20260929 03:02:08.360532 25558 tablet_replica.cc:330] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb: stopping tablet replica
I20260929 03:02:08.361305 25558 raft_consensus.cc:2229] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:02:08.362164 25558 raft_consensus.cc:2258] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:02:08.362454 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.340646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57222 (local address 127.8.208.65:44533)
0929 03:02:08.340860 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:08.340868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:08.340890 (+ 22us) server_negotiation.cc:407] Connection header received
0929 03:02:08.340956 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:08.340965 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:08.341110 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:08.341317 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:08.343783 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.345481 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.348052 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.349023 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.349190 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.354437 (+ 5247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:08.354451 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:08.354469 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:08.354517 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:08.358592 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:08.359472 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:08.359481 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:08.359484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:08.359663 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:08.360150 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:08.360155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:08.360159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:08.360580 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:08.360869 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:08.361779 (+ 910us) server_negotiation.cc:299] Negotiation successful
0929 03:02:08.362011 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 03:02:08.363085 25559 tablet_replica.cc:330] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296: stopping tablet replica
I20260929 03:02:08.363700 25560 tablet_replica.cc:330] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: stopping tablet replica
I20260929 03:02:08.364172 25559 raft_consensus.cc:2229] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:02:08.364642 25559 raft_consensus.cc:2258] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:02:08.364709 25558 ts_tablet_manager.cc:1905] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:02:08.363853 25251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:08.368012 25559 ts_tablet_manager.cc:1905] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:02:08.373023 25558 ts_tablet_manager.cc:1918] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:02:08.373219 25558 log.cc:1196] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/b0e9779f100b4722b7a5fb752f540422
I20260929 03:02:08.373677 25558 ts_tablet_manager.cc:1939] T b0e9779f100b4722b7a5fb752f540422 P 0ce6f1338a894090986afc2f4a6e2bcb: Deleting consensus metadata
I20260929 03:02:08.375285 25055 catalog_manager.cc:4841] TS 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305): tablet b0e9779f100b4722b7a5fb752f540422 (table test-table [id=ce7377b2a01341e08510a7c3f5739131]) successfully deleted
I20260929 03:02:08.375459 25559 ts_tablet_manager.cc:1918] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:02:08.375649 25559 log.cc:1196] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/b0e9779f100b4722b7a5fb752f540422
I20260929 03:02:08.376142 25559 ts_tablet_manager.cc:1939] T b0e9779f100b4722b7a5fb752f540422 P 3aa761b2b7724e83b3185b3343864296: Deleting consensus metadata
I20260929 03:02:08.377882 25056 catalog_manager.cc:4841] TS 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431): tablet b0e9779f100b4722b7a5fb752f540422 (table test-table [id=ce7377b2a01341e08510a7c3f5739131]) successfully deleted
I20260929 03:02:08.365746 25560 raft_consensus.cc:2229] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:02:08.468067 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 25108
I20260929 03:02:08.468149 25560 raft_consensus.cc:2258] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 03:02:08.486181 25056 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.208.65:44533 (error 108)
I20260929 03:02:08.486402 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 25260
I20260929 03:02:08.488142 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.487734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44533 (local address 127.0.0.1:57236)
0929 03:02:08.487904 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.488012 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:44533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260929 03:02:08.488348 25056 catalog_manager.cc:4584] TS 29bb1d4dfb3a4394ad622ffdb6a68ab4 (127.8.208.65:44533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b0e9779f100b4722b7a5fb752f540422: Network error: Client connection negotiation failed: client connection to 127.8.208.65:44533: connect: Connection refused (error 111)
I20260929 03:02:08.504590 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.487178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39830 (local address 127.8.208.66:45305)
0929 03:02:08.487343 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:08.487347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:08.487442 (+ 95us) server_negotiation.cc:407] Connection header received
0929 03:02:08.487599 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:08.487603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:08.487667 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:08.487801 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:08.489282 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.490005 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.491357 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.491770 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.491906 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.498566 (+ 6660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:08.498579 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:08.498594 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:08.498625 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:08.501781 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:08.502480 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:08.502488 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:08.502490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:08.502617 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:08.502998 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:08.503004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:08.503007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:08.503390 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:08.503688 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:08.503983 (+ 295us) server_negotiation.cc:299] Negotiation successful
0929 03:02:08.504223 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 03:02:08.505360 25383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:08.541857 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.541384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44533 (local address 127.0.0.1:57252)
0929 03:02:08.541582 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.541697 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:44533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260929 03:02:08.542930 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.542579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44533 (local address 127.0.0.1:57258)
0929 03:02:08.542727 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.542820 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:44533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 03:02:08.603454 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.603012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44533 (local address 127.0.0.1:57272)
0929 03:02:08.603209 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.603303 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:44533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 03:02:08.604524 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.604194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44533 (local address 127.0.0.1:57286)
0929 03:02:08.604338 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.604413 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:44533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 03:02:08.611393 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 25260
I20260929 03:02:08.624867 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 25391
I20260929 03:02:08.638466 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.625683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33854 (local address 127.8.208.67:35431)
0929 03:02:08.625846 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:08.625851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:08.625869 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:02:08.625960 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:08.625964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:08.626028 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:08.626110 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:08.627393 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.628073 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.630099 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:08.630519 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:08.630630 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:08.633336 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:08.633348 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:08.633364 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:08.633398 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:08.635910 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:08.636535 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:08.636542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:08.636544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:08.636688 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:08.637002 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:08.637005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:08.637007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:08.637550 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:08.637765 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:08.638043 (+ 278us) server_negotiation.cc:299] Negotiation successful
0929 03:02:08.638204 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:02:08.639089 25514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:08.685297 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.684624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44533 (local address 127.0.0.1:57296)
0929 03:02:08.684935 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.685038 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:44533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260929 03:02:08.686529 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.686092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44533 (local address 127.0.0.1:57310)
0929 03:02:08.686304 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.686400 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:44533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260929 03:02:08.728628 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 25391
I20260929 03:02:08.745409 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 25038
I20260929 03:02:08.746589 25101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:08.861341 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 25038
I20260929 03:02:08.861460 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:08.860898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44533 (local address 127.0.0.1:57322)
0929 03:02:08.861112 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:08.861246 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:44533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260929 03:02:28.882946 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37897
--webserver_interface=127.8.208.126
--webserver_port=38337
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:37897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:02:29.213814 25566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:29.214116 25566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:29.214272 25566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:29.221606 25566 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 03:02:29.221695 25566 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 03:02:29.221766 25566 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 03:02:29.221841 25566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:02:29.221947 25566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:29.222071 25566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:29.222133 25566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:02:29.222227 25566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:02:29.232095 25566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:37897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:37897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=38337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:29.233443 25566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:29.234766 25566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:29.247442 25572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:29.247423 25574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:29.247563 25571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:29.248114 25566 server_base.cc:1034] running on GCE node
I20260929 03:02:29.249006 25566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:29.250574 25566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:29.251819 25566 hybrid_clock.cc:648] HybridClock initialized: now 1790650949251746 us; error 40 us; skew 500 ppm
I20260929 03:02:29.252251 25566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:29 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650949, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:29.262776 25566 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:02:29.264349 25566 webserver.cc:460] Webserver started at http://127.8.208.126:38337/ using document root <none> and password file <none>
I20260929 03:02:29.265053 25566 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:29.265209 25566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:29.270602 25566 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:02:29.273381 25581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:29.274591 25566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:02:29.274768 25566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "76455a2e30954eb5a0e057e624da58c9"
format_stamp: "Formatted at 2026-09-29 03:01:43 on dist-test-slave-m1rv"
server_key: "139a53459a770f13f2455028582f387b"
server_key_iv: "e45ac597e02bbf88cf771345701294fa"
server_key_version: "encryptionkey@0"
I20260929 03:02:29.275697 25566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:29.313413 25566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:29.317632 25566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:29.317924 25566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:29.336445 25566 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:37897
I20260929 03:02:29.336540 25632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:37897 every 8 connection(s)
I20260929 03:02:29.338105 25566 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:02:29.345062 25633 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:02:29.347325 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 25566
I20260929 03:02:29.348169 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:44533
--local_ip_for_outbound_sockets=127.8.208.65
--tserver_master_addrs=127.8.208.126:37897
--webserver_port=44589
--webserver_interface=127.8.208.65
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:02:29.348428 25633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Bootstrap starting.
I20260929 03:02:29.368933 25633 log.cc:824] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:29.385850 25633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:02:29.386430 25633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Bootstrap complete.
I20260929 03:02:29.395838 25633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } }
I20260929 03:02:29.396785 25633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76455a2e30954eb5a0e057e624da58c9, State: Initialized, Role: FOLLOWER
I20260929 03:02:29.397426 25633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } }
I20260929 03:02:29.397686 25633 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:02:29.397823 25633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:02:29.398121 25633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 2 FOLLOWER]: Advancing to term 3
I20260929 03:02:29.402030 25633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } }
I20260929 03:02:29.402561 25633 leader_election.cc:302] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76455a2e30954eb5a0e057e624da58c9; no voters:
I20260929 03:02:29.403304 25633 leader_election.cc:288] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 03:02:29.403469 25637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 03:02:29.404804 25637 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [term 3 LEADER]: Becoming Leader. State: Replica: 76455a2e30954eb5a0e057e624da58c9, State: Running, Role: LEADER
I20260929 03:02:29.405475 25637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } }
I20260929 03:02:29.405797 25633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:02:29.407968 25639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76455a2e30954eb5a0e057e624da58c9. Latest consensus state: current_term: 3 leader_uuid: "76455a2e30954eb5a0e057e624da58c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } } }
I20260929 03:02:29.408324 25639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: This master's current role is: LEADER
I20260929 03:02:29.408834 25638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "76455a2e30954eb5a0e057e624da58c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76455a2e30954eb5a0e057e624da58c9" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 37897 } } }
I20260929 03:02:29.409107 25638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9 [sys.catalog]: This master's current role is: LEADER
I20260929 03:02:29.413535 25643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:02:29.424034 25643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:02:29.426261 25643 catalog_manager.cc:1259] Loaded cluster ID: d58f31b4d917450cb3d3e69bb0050c01
I20260929 03:02:29.426487 25643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:02:29.431840 25643 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:02:29.434332 25643 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Loaded TSK: 0
I20260929 03:02:29.434664 25643 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 76455a2e30954eb5a0e057e624da58c9: Loaded TSK: 1
F20260929 03:02:29.436133 25643 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f41990599ad google::LogMessage::Fail() at ??:0
@ 0x7f419905c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f41990594de google::LogMessage::Flush() at ??:0
@ 0x7f419905aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f419cdbd7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f419cdac73e kudu::security::TokenSigner::ImportKeys() at ??:0
I20260929 03:02:29.460263 25656 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f41aea881c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f41aea87783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f41aeb132a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f41aeb130c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f41aeb5d5e9 std::function<>::operator()() at ??:0
@ 0x7f41aea8bcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f41aea89c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f41aeb11610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f41aeb11439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f419bca5345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f419bca891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f419bca8799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f419bc83eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f419ef5d6db start_thread at ??:0
@ 0x7f4197ef361f clone at ??:0
W20260929 03:02:29.727210 25635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:29.727571 25635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:29.727728 25635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:29.735163 25635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:29.735339 25635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:29.735534 25635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:02:29.745561 25635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:44533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=44589
--tserver_master_addrs=127.8.208.126:37897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:29.746868 25635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:29.748210 25635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:29.761368 25680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:29.761440 25681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:29.761895 25683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:29.762470 25635 server_base.cc:1034] running on GCE node
I20260929 03:02:29.763090 25635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:29.764061 25635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:29.765309 25635 hybrid_clock.cc:648] HybridClock initialized: now 1790650949765250 us; error 58 us; skew 500 ppm
I20260929 03:02:29.765666 25635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:29 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650949, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:29.768949 25635 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 03:02:29.770511 25635 webserver.cc:460] Webserver started at http://127.8.208.65:44589/ using document root <none> and password file <none>
I20260929 03:02:29.771224 25635 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:29.771392 25635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:29.776890 25635 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 03:02:29.779685 25690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:29.781050 25635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:02:29.781260 25635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4"
format_stamp: "Formatted at 2026-09-29 03:01:43 on dist-test-slave-m1rv"
server_key: "f49af919cb0c868fd412e9b32f117cdd"
server_key_iv: "76274b4cb60b1a98852840253563c78e"
server_key_version: "encryptionkey@0"
I20260929 03:02:29.782208 25635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:29.796766 25635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:29.801160 25635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:29.801472 25635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:29.802433 25635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:29.804158 25697 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260929 03:02:29.809688 25635 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 03:02:29.809845 25635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260929 03:02:29.809989 25635 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 03:02:29.813072 25635 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 03:02:29.813199 25635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260929 03:02:29.813470 25697 tablet_bootstrap.cc:492] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Bootstrap starting.
I20260929 03:02:29.840582 25697 log.cc:824] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:29.855603 25635 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:44533
I20260929 03:02:29.855623 25804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:44533 every 8 connection(s)
I20260929 03:02:29.857218 25635 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:02:29.860028 25697 tablet_bootstrap.cc:492] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:02:29.860574 25697 tablet_bootstrap.cc:492] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Bootstrap complete.
I20260929 03:02:29.861943 25697 ts_tablet_manager.cc:1397] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Time spent bootstrapping tablet: real 0.049s user 0.037s sys 0.011s
I20260929 03:02:29.862130 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 25635
I20260929 03:02:29.862371 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:29.859933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:37687)
0929 03:02:29.860883 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:29.861200 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":450,"threads_started":1}
I20260929 03:02:29.862914 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:45305
--local_ip_for_outbound_sockets=127.8.208.66
--tserver_master_addrs=127.8.208.126:37897
--webserver_port=41895
--webserver_interface=127.8.208.66
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:02:29.863209 25805 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:37897 (0 consecutive failures): Network error: Failed to ping master at 127.8.208.126:37897: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
I20260929 03:02:29.864203 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:29.863788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:43289)
0929 03:02:29.863973 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:29.864067 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 03:02:29.865374 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:29.864897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:36881)
0929 03:02:29.865059 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:29.865160 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260929 03:02:29.865689 25805 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 03:02:29.869123 25697 raft_consensus.cc:348] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:29.870150 25697 raft_consensus.cc:729] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29bb1d4dfb3a4394ad622ffdb6a68ab4, State: Initialized, Role: FOLLOWER
I20260929 03:02:29.870803 25697 consensus_queue.cc:260] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:29.874794 25697 ts_tablet_manager.cc:1428] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260929 03:02:29.875247 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:29.874595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:60347)
0929 03:02:29.874866 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:29.875083 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260929 03:02:29.877223 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:29.876713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:55099)
0929 03:02:29.876932 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:29.877029 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 03:02:30.194167 25808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:30.194486 25808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:30.194612 25808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:30.202100 25808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:30.202206 25808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:30.202318 25808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:02:30.212073 25808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:45305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=41895
--tserver_master_addrs=127.8.208.126:37897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:30.213346 25808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:30.214601 25808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:30.227344 25814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:30.227432 25815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:30.227401 25817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:30.228324 25808 server_base.cc:1034] running on GCE node
I20260929 03:02:30.228854 25808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:30.229808 25808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:30.231004 25808 hybrid_clock.cc:648] HybridClock initialized: now 1790650950230951 us; error 54 us; skew 500 ppm
I20260929 03:02:30.231303 25808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:30 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650950, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:30.234503 25808 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 03:02:30.235925 25808 webserver.cc:460] Webserver started at http://127.8.208.66:41895/ using document root <none> and password file <none>
I20260929 03:02:30.236591 25808 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:30.236752 25808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:30.241880 25808 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:02:30.244405 25824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:30.245748 25808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:02:30.245920 25808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "0ce6f1338a894090986afc2f4a6e2bcb"
format_stamp: "Formatted at 2026-09-29 03:01:44 on dist-test-slave-m1rv"
server_key: "0dc7ef68625c8f435fbac0b944c03f89"
server_key_iv: "a19f197e47bc37ff66201f68186bc9dd"
server_key_version: "encryptionkey@0"
I20260929 03:02:30.246860 25808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:30.262446 25808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:30.266345 25808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:30.266619 25808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:30.267627 25808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:30.268900 25808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:30.269024 25808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:30.269158 25808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:30.269266 25808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260929 03:02:30.303521 25808 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:45305
I20260929 03:02:30.303613 25936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:45305 every 8 connection(s)
I20260929 03:02:30.305038 25808 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:02:30.309676 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 25808
I20260929 03:02:30.310189 25939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:30.308278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:53349)
0929 03:02:30.309002 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:30.309289 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":303,"threads_started":1}
I20260929 03:02:30.310411 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:35431
--local_ip_for_outbound_sockets=127.8.208.67
--tserver_master_addrs=127.8.208.126:37897
--webserver_port=45479
--webserver_interface=127.8.208.67
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:38535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:02:30.311149 25937 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:37897 (0 consecutive failures): Network error: Failed to ping master at 127.8.208.126:37897: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
I20260929 03:02:30.312000 25939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:30.311679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:50377)
0929 03:02:30.311834 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:30.311906 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 03:02:30.312918 25939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:30.312590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:40677)
0929 03:02:30.312732 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:30.312814 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 03:02:30.313203 25937 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 03:02:30.634451 25940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:30.634718 25940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:30.634811 25940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:30.641939 25940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:30.642035 25940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:30.642143 25940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:02:30.651686 25940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:38535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:35431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=45479
--tserver_master_addrs=127.8.208.126:37897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:30.652906 25940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:30.654150 25940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:30.666564 25948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:30.666728 25945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:30.666662 25946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:30.667326 25940 server_base.cc:1034] running on GCE node
I20260929 03:02:30.667838 25940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:30.668648 25940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:30.669873 25940 hybrid_clock.cc:648] HybridClock initialized: now 1790650950669831 us; error 55 us; skew 500 ppm
I20260929 03:02:30.670190 25940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:30 dist-test-slave-m1rv krb5kdc[24487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650950, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:30.673113 25940 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 03:02:30.674434 25940 webserver.cc:460] Webserver started at http://127.8.208.67:45479/ using document root <none> and password file <none>
I20260929 03:02:30.675028 25940 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:30.675125 25940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:30.680140 25940 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260929 03:02:30.683000 25955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:30.684291 25940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260929 03:02:30.684485 25940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "3aa761b2b7724e83b3185b3343864296"
format_stamp: "Formatted at 2026-09-29 03:01:44 on dist-test-slave-m1rv"
server_key: "446c26daa59bc3ef0b97154855a829a5"
server_key_iv: "90a19db8199d990fdb869ff722c81583"
server_key_version: "encryptionkey@0"
I20260929 03:02:30.685499 25940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:30.707909 25940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:30.711946 25940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:30.712234 25940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:30.713258 25940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:30.714589 25940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:30.714675 25940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:30.714838 25940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:30.714931 25940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:30.748111 25940 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:35431
I20260929 03:02:30.748203 26067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:35431 every 8 connection(s)
I20260929 03:02:30.749691 25940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:02:30.753885 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 25940
I20260929 03:02:30.755295 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:30.753242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:55703)
0929 03:02:30.754112 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:30.754414 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":373,"threads_started":1}
W20260929 03:02:30.755949 26068 heartbeater.cc:644] Failed to heartbeat to 127.8.208.126:37897 (0 consecutive failures): Network error: Failed to ping master at 127.8.208.126:37897: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
I20260929 03:02:30.756805 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:30.756417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:60219)
0929 03:02:30.756588 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:30.756680 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260929 03:02:30.757838 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:30.757490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:53795)
0929 03:02:30.757624 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:30.757708 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260929 03:02:30.758145 26068 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 03:02:30.878752 26072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:30.878021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:51333)
0929 03:02:30.878447 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:30.878538 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":223,"threads_started":1}
I20260929 03:02:31.314786 26073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:31.313963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:53099)
0929 03:02:31.314457 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:31.314566 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":254,"threads_started":1}
I20260929 03:02:31.716260 26074 raft_consensus.cc:482] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:02:31.716614 26074 raft_consensus.cc:504] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:31.718199 26074 leader_election.cc:288] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305), 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431)
Sep 29 03:02:31 dist-test-slave-m1rv krb5kdc[24487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650949, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
Sep 29 03:02:31 dist-test-slave-m1rv krb5kdc[24487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650949, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 03:02:31.750229 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:31.718620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35431 (local address 127.8.208.65:49963)
0929 03:02:31.730875 (+ 12255us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:31.730890 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:31.731370 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:31.733856 (+ 2486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:31.733868 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:31.734786 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:31.735483 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:31.735496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:31.736997 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:31.737001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:31.738706 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:31.738717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:31.739664 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:31.739670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:31.739926 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:31.740736 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:02:31.740753 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:31.743380 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:31.746648 (+ 3268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:31.746654 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:31.746666 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:31.747118 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:31.747581 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:31.747590 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:31.747594 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:31.747830 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:31.748583 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:31.748592 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:31.748926 (+ 334us) client_negotiation.cc:769] Sending connection context
0929 03:02:31.749275 (+ 349us) client_negotiation.cc:240] Negotiation successful
0929 03:02:31.749724 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12161,"mutex_wait_us":121,"spinlock_wait_cycles":42112,"thread_start_us":11983,"threads_started":1}
I20260929 03:02:31.750367 26078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:31.718839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:46453 (local address 127.8.208.66:45305)
0929 03:02:31.729564 (+ 10725us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:31.729589 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:31.731034 (+ 1445us) server_negotiation.cc:407] Connection header received
0929 03:02:31.733066 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:31.733074 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:31.733441 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:31.733743 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:31.735667 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:31.736801 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:31.738833 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:31.739432 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:31.739630 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:31.743316 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:31.743323 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:31.743335 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:31.743371 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:31.746402 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:31.747246 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:31.747253 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:31.747276 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:31.747472 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:31.747972 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:31.747977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:31.747980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:31.748606 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:31.748792 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:31.749525 (+ 733us) server_negotiation.cc:299] Negotiation successful
0929 03:02:31.749926 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10464,"thread_start_us":10773,"threads_started":1}
I20260929 03:02:31.750980 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:31.718846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:49963 (local address 127.8.208.67:35431)
0929 03:02:31.730315 (+ 11469us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:31.730335 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:31.731105 (+ 770us) server_negotiation.cc:407] Connection header received
0929 03:02:31.732988 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:31.732998 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:31.733415 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:31.733743 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:31.735660 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:31.736794 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:31.738832 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:31.739450 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:31.739650 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:31.743632 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:31.743640 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:31.743651 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:31.743683 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:31.746516 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:31.747255 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:31.747262 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:31.747266 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:31.747421 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:31.747945 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:31.747950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:31.747953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:31.748420 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:31.748651 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:31.750068 (+ 1417us) server_negotiation.cc:299] Negotiation successful
0929 03:02:31.750503 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11259,"thread_start_us":11100,"threads_started":1}
I20260929 03:02:31.751245 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:31.718539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:45305 (local address 127.8.208.65:46453)
0929 03:02:31.730845 (+ 12306us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:31.730889 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:31.731391 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:31.733857 (+ 2466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:31.733869 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:31.734759 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:31.735467 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:31.735486 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:31.736984 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:31.736988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:31.738704 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:31.738716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:31.739644 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:31.739650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:31.739936 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:31.740663 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:02:31.740692 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:31.743045 (+ 2353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:31.746587 (+ 3542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:31.746594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:31.746608 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:31.747100 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:31.747629 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:31.747633 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:31.747636 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:31.747844 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:31.748860 (+ 1016us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:31.748865 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:31.749018 (+ 153us) client_negotiation.cc:769] Sending connection context
0929 03:02:31.750545 (+ 1527us) client_negotiation.cc:240] Negotiation successful
0929 03:02:31.750789 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12151,"mutex_wait_us":171,"spinlock_wait_cycles":15232,"thread_start_us":11814,"threads_started":1}
W20260929 03:02:31.754477 25691 leader_election.cc:341] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305): Not found: Tablet not found: b0e9779f100b4722b7a5fb752f540422
W20260929 03:02:31.755537 25692 leader_election.cc:341] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431): Not found: Tablet not found: b0e9779f100b4722b7a5fb752f540422
I20260929 03:02:31.755647 25692 leader_election.cc:302] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29bb1d4dfb3a4394ad622ffdb6a68ab4; no voters: 0ce6f1338a894090986afc2f4a6e2bcb, 3aa761b2b7724e83b3185b3343864296
I20260929 03:02:31.756067 26074 raft_consensus.cc:2726] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 03:02:31.759353 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:31.758575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:51823)
0929 03:02:31.759015 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:31.759157 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":207,"threads_started":1}
I20260929 03:02:31.880402 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:31.879884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:43599)
0929 03:02:31.880107 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:31.880214 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260929 03:02:32.316679 26080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:32.315882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:59161)
0929 03:02:32.316396 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:32.316495 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":268,"threads_started":1}
I20260929 03:02:32.761004 26081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:32.760303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:54143)
0929 03:02:32.760770 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:32.760858 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":253,"threads_started":1}
I20260929 03:02:32.882609 26082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:32.881676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:38473)
0929 03:02:32.882307 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:32.882426 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":258,"threads_started":1}
I20260929 03:02:33.318336 26083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:33.317646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:42885)
0929 03:02:33.318086 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:33.318167 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":222,"threads_started":1}
I20260929 03:02:33.668107 26084 raft_consensus.cc:482] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:02:33.668294 26084 raft_consensus.cc:504] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:33.668987 26084 leader_election.cc:288] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305), 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431)
W20260929 03:02:33.669993 25692 leader_election.cc:341] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431): Not found: Tablet not found: b0e9779f100b4722b7a5fb752f540422
W20260929 03:02:33.670185 25691 leader_election.cc:341] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305): Not found: Tablet not found: b0e9779f100b4722b7a5fb752f540422
I20260929 03:02:33.670323 25691 leader_election.cc:302] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29bb1d4dfb3a4394ad622ffdb6a68ab4; no voters: 0ce6f1338a894090986afc2f4a6e2bcb, 3aa761b2b7724e83b3185b3343864296
I20260929 03:02:33.670624 26084 raft_consensus.cc:2726] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 03:02:33.762564 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:33.761901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:48477)
0929 03:02:33.762326 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:33.762407 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":221,"threads_started":1}
I20260929 03:02:33.884356 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:33.883575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:51329)
0929 03:02:33.884058 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:33.884165 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":217,"threads_started":1}
I20260929 03:02:34.320228 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:34.319256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:47611)
0929 03:02:34.319842 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:34.319981 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":346,"threads_started":1}
I20260929 03:02:34.764086 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:34.763419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:57589)
0929 03:02:34.763831 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:34.763910 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":210,"threads_started":1}
I20260929 03:02:34.886365 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:34.885606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:34189)
0929 03:02:34.886084 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:34.886175 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":218,"threads_started":1}
I20260929 03:02:35.321890 26090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:35.321225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:36449)
0929 03:02:35.321651 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:35.321728 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":227,"threads_started":1}
I20260929 03:02:35.765866 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:35.765054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:44529)
0929 03:02:35.765541 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:35.765656 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":289,"threads_started":1}
I20260929 03:02:35.888041 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:35.887253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:36781)
0929 03:02:35.887755 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:35.887842 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":254,"threads_started":1}
I20260929 03:02:36.323501 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:36.322776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:45417)
0929 03:02:36.323214 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:36.323298 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":236,"threads_started":1}
I20260929 03:02:36.767474 26094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:36.766763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:45881)
0929 03:02:36.767220 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:36.767302 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":238,"threads_started":1}
I20260929 03:02:36.889690 26095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:36.888929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:47137)
0929 03:02:36.889413 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:36.889499 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":292,"threads_started":1}
I20260929 03:02:37.043303 26096 raft_consensus.cc:482] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:02:37.043515 26096 raft_consensus.cc:504] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:37.044239 26096 leader_election.cc:288] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305), 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431)
W20260929 03:02:37.045121 25691 leader_election.cc:341] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305): Not found: Tablet not found: b0e9779f100b4722b7a5fb752f540422
W20260929 03:02:37.045328 25692 leader_election.cc:341] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431): Not found: Tablet not found: b0e9779f100b4722b7a5fb752f540422
I20260929 03:02:37.045495 25692 leader_election.cc:302] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29bb1d4dfb3a4394ad622ffdb6a68ab4; no voters: 0ce6f1338a894090986afc2f4a6e2bcb, 3aa761b2b7724e83b3185b3343864296
I20260929 03:02:37.045745 26096 raft_consensus.cc:2726] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 03:02:37.325631 26097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:37.324579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:55463)
0929 03:02:37.325351 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:37.325448 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":382,"threads_started":1}
I20260929 03:02:37.769125 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:37.768359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:34723)
0929 03:02:37.768831 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:37.768911 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":214,"threads_started":1}
I20260929 03:02:37.891168 26099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:37.890497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:49147)
0929 03:02:37.890912 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:37.891015 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":204,"threads_started":1}
I20260929 03:02:38.327247 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:38.326536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:43087)
0929 03:02:38.326953 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:38.327075 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":215,"threads_started":1}
I20260929 03:02:38.770856 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:38.770129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:38637)
0929 03:02:38.770584 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:38.770675 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":226,"threads_started":1}
I20260929 03:02:38.893009 26102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:38.892227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:47489)
0929 03:02:38.892755 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:38.892836 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":302,"threads_started":1}
I20260929 03:02:39.329064 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:39.328119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:48485)
0929 03:02:39.328712 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:39.328819 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":281,"threads_started":1}
I20260929 03:02:39.460647 26104 raft_consensus.cc:482] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:02:39.460865 26104 raft_consensus.cc:504] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:39.461617 26104 leader_election.cc:288] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305), 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431)
W20260929 03:02:39.462561 25692 leader_election.cc:341] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431): Not found: Tablet not found: b0e9779f100b4722b7a5fb752f540422
W20260929 03:02:39.462756 25691 leader_election.cc:341] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305): Not found: Tablet not found: b0e9779f100b4722b7a5fb752f540422
I20260929 03:02:39.462860 25691 leader_election.cc:302] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29bb1d4dfb3a4394ad622ffdb6a68ab4; no voters: 0ce6f1338a894090986afc2f4a6e2bcb, 3aa761b2b7724e83b3185b3343864296
I20260929 03:02:39.463066 26104 raft_consensus.cc:2726] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 03:02:39.772468 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:39.771788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:41831)
0929 03:02:39.772234 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:39.772319 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":230,"threads_started":1}
I20260929 03:02:39.894870 26106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:39.894026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:51589)
0929 03:02:39.894525 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:39.894653 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":279,"threads_started":1}
I20260929 03:02:40.330945 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:40.330172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:55547)
0929 03:02:40.330699 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:40.330776 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":319,"threads_started":1}
I20260929 03:02:40.774312 26108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:40.773533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:36067)
0929 03:02:40.774062 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:40.774139 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":317,"threads_started":1}
I20260929 03:02:40.896488 26109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:40.895773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:46293)
0929 03:02:40.896209 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:40.896289 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":229,"threads_started":1}
I20260929 03:02:41.332584 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:41.331814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:33915)
0929 03:02:41.332312 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:41.332393 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":259,"threads_started":1}
I20260929 03:02:41.775941 26111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:41.775217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:46815)
0929 03:02:41.775692 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:41.775774 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":223,"threads_started":1}
I20260929 03:02:41.898240 26112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:41.897546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:52785)
0929 03:02:41.897991 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:41.898081 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":236,"threads_started":1}
I20260929 03:02:42.334353 26113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:42.333640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:33707)
0929 03:02:42.334098 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:42.334181 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":233,"threads_started":1}
I20260929 03:02:42.777900 26114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:42.777114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:34957)
0929 03:02:42.777636 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:42.777736 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":315,"threads_started":1}
I20260929 03:02:42.900054 26115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:42.899189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:41475)
0929 03:02:42.899678 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:42.899780 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":249,"threads_started":1}
I20260929 03:02:43.014720 26116 raft_consensus.cc:482] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:02:43.014919 26116 raft_consensus.cc:504] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ce6f1338a894090986afc2f4a6e2bcb" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 45305 } } peers { permanent_uuid: "3aa761b2b7724e83b3185b3343864296" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35431 } } peers { permanent_uuid: "29bb1d4dfb3a4394ad622ffdb6a68ab4" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44533 } }
I20260929 03:02:43.015692 26116 leader_election.cc:288] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305), 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431)
W20260929 03:02:43.016767 25692 leader_election.cc:341] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3aa761b2b7724e83b3185b3343864296 (127.8.208.67:35431): Not found: Tablet not found: b0e9779f100b4722b7a5fb752f540422
W20260929 03:02:43.016978 25691 leader_election.cc:341] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ce6f1338a894090986afc2f4a6e2bcb (127.8.208.66:45305): Not found: Tablet not found: b0e9779f100b4722b7a5fb752f540422
I20260929 03:02:43.017096 25691 leader_election.cc:302] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 29bb1d4dfb3a4394ad622ffdb6a68ab4; no voters: 0ce6f1338a894090986afc2f4a6e2bcb, 3aa761b2b7724e83b3185b3343864296
I20260929 03:02:43.017447 26116 raft_consensus.cc:2726] T b0e9779f100b4722b7a5fb752f540422 P 29bb1d4dfb3a4394ad622ffdb6a68ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 03:02:43.336133 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:43.335323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:36395)
0929 03:02:43.335863 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:43.335970 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":257,"threads_started":1}
I20260929 03:02:43.779560 26118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:43.778833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:58659)
0929 03:02:43.779310 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:43.779399 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":270,"threads_started":1}
I20260929 03:02:43.901638 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:43.900928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:48269)
0929 03:02:43.901364 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:43.901442 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":268,"threads_started":1}
I20260929 03:02:44.337965 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:44.337145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:47975)
0929 03:02:44.337690 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:44.337773 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":227,"threads_started":1}
I20260929 03:02:44.781329 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:44.780521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:53921)
0929 03:02:44.781035 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:44.781119 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":251,"threads_started":1}
I20260929 03:02:44.903509 26122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:44.902780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.65:59303)
0929 03:02:44.903247 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:44.903335 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":241,"threads_started":1}
I20260929 03:02:45.339784 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:45.339058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.66:46277)
0929 03:02:45.339500 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:45.339620 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":216,"threads_started":1}
I20260929 03:02:45.762847 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 25635
I20260929 03:02:45.780068 26124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:45.764199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57802 (local address 127.8.208.65:44533)
0929 03:02:45.764881 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:45.764890 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:45.765125 (+ 235us) server_negotiation.cc:407] Connection header received
0929 03:02:45.765566 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:45.765579 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:45.765844 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:45.766049 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:45.767840 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:45.768943 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:45.770468 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:45.771119 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:45.771287 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:45.774316 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:45.774328 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:45.774342 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:45.774390 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:45.777164 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:45.777924 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:45.777931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:45.777934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:45.778076 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:45.778462 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:45.778466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:45.778468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:45.778815 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:45.779134 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:45.779457 (+ 323us) server_negotiation.cc:299] Negotiation successful
0929 03:02:45.779760 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":320,"threads_started":1}
I20260929 03:02:45.781345 25800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:45.783690 26126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:45.782724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:37897 (local address 127.8.208.67:44763)
0929 03:02:45.783269 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:45.783428 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.126:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":235,"threads_started":1}
I20260929 03:02:45.871553 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 25635
I20260929 03:02:45.892021 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 25808
I20260929 03:02:45.905709 26127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:45.892884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60810 (local address 127.8.208.66:45305)
0929 03:02:45.893445 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:45.893450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:45.893466 (+ 16us) server_negotiation.cc:407] Connection header received
0929 03:02:45.893523 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:45.893527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:45.893587 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:45.893679 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:45.894969 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:45.895774 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:45.897013 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:45.897563 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:45.897668 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:45.900194 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:45.900199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:45.900202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:45.900235 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:45.903019 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:45.903743 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:45.903747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:45.903750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:45.903878 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:45.904239 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:45.904243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:45.904244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:45.904510 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:45.904717 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:45.905068 (+ 351us) server_negotiation.cc:299] Negotiation successful
0929 03:02:45.905303 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":343,"threads_started":1}
I20260929 03:02:45.906600 25932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:45.991250 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 25808
I20260929 03:02:46.008628 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 25940
I20260929 03:02:46.021723 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:46.009531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35954 (local address 127.8.208.67:35431)
0929 03:02:46.010108 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:46.010112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:46.010129 (+ 17us) server_negotiation.cc:407] Connection header received
0929 03:02:46.010182 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:46.010186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:46.010260 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:46.010351 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:46.011679 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:46.012490 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:46.013820 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:46.014246 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:46.014342 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:46.016833 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:46.016837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:46.016841 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:46.016871 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:46.019440 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:46.020105 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:46.020109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:46.020111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:46.020232 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:46.020585 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:46.020588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:46.020590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:46.020844 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:46.021042 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:46.021270 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 03:02:46.021433 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":319,"threads_started":1}
I20260929 03:02:46.022414 26063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:46.106220 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 25940
2026-09-29T03:02:46Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64551 ms)
[----------] 30 tests from SecurityITest (293492 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:02:46 dist-test-slave-m1rv krb5kdc[26133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:02:46 dist-test-slave-m1rv krb5kdc[26133](info): set up 2 sockets
Sep 29 03:02:46 dist-test-slave-m1rv krb5kdc[26133](info): commencing operation
krb5kdc: starting...
W20260929 03:02:47.229491 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:02:47 dist-test-slave-m1rv krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:02:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:02:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:02:47.476290 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:46543
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:43135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:46543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:02:47.806350 26146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:47.806638 26146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:47.806816 26146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:47.814059 26146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:02:47.814157 26146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:47.814275 26146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:47.814370 26146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:02:47.814447 26146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:02:47.824276 26146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:43135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:46543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:46543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:47.825592 26146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:47.826910 26146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:47.839020 26151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:47.839110 26154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:47.839138 26152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:47.839350 26146 server_base.cc:1034] running on GCE node
I20260929 03:02:47.840368 26146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:47.841831 26146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:47.843098 26146 hybrid_clock.cc:648] HybridClock initialized: now 1790650967843031 us; error 44 us; skew 500 ppm
I20260929 03:02:47.843443 26146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:47 dist-test-slave-m1rv krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650967, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:47.851933 26146 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:02:47.853477 26146 webserver.cc:460] Webserver started at http://127.8.208.126:41253/ using document root <none> and password file <none>
I20260929 03:02:47.854144 26146 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:47.854282 26146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:47.854578 26146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:47.857560 26146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "d211c56def714fd0a6586a51227e90d4"
format_stamp: "Formatted at 2026-09-29 03:02:47 on dist-test-slave-m1rv"
server_key: "debbc65936dfb79d96155d8c213f86f9"
server_key_iv: "929b51372e380a264624222bb0258cdf"
server_key_version: "encryptionkey@0"
I20260929 03:02:47.858284 26146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "d211c56def714fd0a6586a51227e90d4"
format_stamp: "Formatted at 2026-09-29 03:02:47 on dist-test-slave-m1rv"
server_key: "debbc65936dfb79d96155d8c213f86f9"
server_key_iv: "929b51372e380a264624222bb0258cdf"
server_key_version: "encryptionkey@0"
I20260929 03:02:47.863250 26146 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 03:02:47.866639 26161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:47.867848 26146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:02:47.868010 26146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "d211c56def714fd0a6586a51227e90d4"
format_stamp: "Formatted at 2026-09-29 03:02:47 on dist-test-slave-m1rv"
server_key: "debbc65936dfb79d96155d8c213f86f9"
server_key_iv: "929b51372e380a264624222bb0258cdf"
server_key_version: "encryptionkey@0"
I20260929 03:02:47.868211 26146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:47.890942 26146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:47.895226 26146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:47.895586 26146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:47.913805 26146 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:46543
I20260929 03:02:47.913831 26212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:46543 every 8 connection(s)
I20260929 03:02:47.915155 26146 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:02:47.918957 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 26146
I20260929 03:02:47.919224 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:02:47.919581 26213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:47.920100 9025 external_mini_cluster.cc:1428] Setting key f491ec731cf59db7bc3f77a60b15acd3
Sep 29 03:02:47 dist-test-slave-m1rv krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:02:47.935796 26213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4: Bootstrap starting.
I20260929 03:02:47.939429 26213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:47.940621 26213 log.cc:824] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:47.944134 26213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4: No bootstrap required, opened a new log
I20260929 03:02:47.944700 26216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:47.922318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46722 (local address 127.8.208.126:46543)
0929 03:02:47.923121 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:47.923135 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:47.923177 (+ 42us) server_negotiation.cc:407] Connection header received
0929 03:02:47.924832 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:47.924853 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:47.925284 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:47.925626 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:47.927646 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:47.928923 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:47.931051 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:47.931745 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:47.931878 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:47.937215 (+ 5337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:47.937229 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:47.937249 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:47.937289 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:47.940365 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:47.941279 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:47.941294 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:47.941303 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:47.941554 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:47.941960 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:47.941965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:47.941968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:47.942484 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:47.942694 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:47.943310 (+ 616us) server_negotiation.cc:299] Negotiation successful
0929 03:02:47.943646 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":402,"threads_started":1}
I20260929 03:02:47.952152 26213 raft_consensus.cc:348] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d211c56def714fd0a6586a51227e90d4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46543 } }
I20260929 03:02:47.952515 26213 raft_consensus.cc:374] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:47.952620 26213 raft_consensus.cc:729] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d211c56def714fd0a6586a51227e90d4, State: Initialized, Role: FOLLOWER
I20260929 03:02:47.953258 26213 consensus_queue.cc:260] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d211c56def714fd0a6586a51227e90d4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46543 } }
I20260929 03:02:47.953495 26213 raft_consensus.cc:388] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:02:47.953655 26213 raft_consensus.cc:482] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:02:47.953840 26213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:47.956149 26213 raft_consensus.cc:504] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d211c56def714fd0a6586a51227e90d4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46543 } }
I20260929 03:02:47.956667 26213 leader_election.cc:302] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d211c56def714fd0a6586a51227e90d4; no voters:
I20260929 03:02:47.957305 26213 leader_election.cc:288] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:02:47.957441 26218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:02:47.958189 26218 raft_consensus.cc:686] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [term 1 LEADER]: Becoming Leader. State: Replica: d211c56def714fd0a6586a51227e90d4, State: Running, Role: LEADER
I20260929 03:02:47.958741 26218 consensus_queue.cc:237] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d211c56def714fd0a6586a51227e90d4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46543 } }
I20260929 03:02:47.959450 26213 sys_catalog.cc:564] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:02:47.961428 26219 sys_catalog.cc:455] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d211c56def714fd0a6586a51227e90d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d211c56def714fd0a6586a51227e90d4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46543 } } }
I20260929 03:02:47.961212 26220 sys_catalog.cc:455] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d211c56def714fd0a6586a51227e90d4. Latest consensus state: current_term: 1 leader_uuid: "d211c56def714fd0a6586a51227e90d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d211c56def714fd0a6586a51227e90d4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46543 } } }
I20260929 03:02:47.961804 26219 sys_catalog.cc:458] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [sys.catalog]: This master's current role is: LEADER
I20260929 03:02:47.961812 26220 sys_catalog.cc:458] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4 [sys.catalog]: This master's current role is: LEADER
I20260929 03:02:47.965631 26225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:02:47.971331 26225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:02:47.980237 26225 catalog_manager.cc:1347] Generated new cluster ID: 714ebdb7890d4d6080c12d6e35741081
I20260929 03:02:47.980342 26225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:02:47.995654 26225 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:02:47.997071 26225 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:02:48.003705 26225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4: Generated new TSK 0
I20260929 03:02:48.004525 26225 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 03:02:48.072443 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46543
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:43135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:02:48.405476 26239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:48.405748 26239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:48.405921 26239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:48.413261 26239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:48.413393 26239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:48.413566 26239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:02:48.423343 26239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:43135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46543
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:48.424682 26239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:48.426170 26239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:48.439236 26244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:48.439374 26247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:48.439445 26245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:48.440558 26239 server_base.cc:1034] running on GCE node
I20260929 03:02:48.441156 26239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:48.442150 26239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:48.443382 26239 hybrid_clock.cc:648] HybridClock initialized: now 1790650968443310 us; error 64 us; skew 500 ppm
I20260929 03:02:48.443684 26239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:48 dist-test-slave-m1rv krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650968, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:48.446867 26239 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 03:02:48.448230 26239 webserver.cc:460] Webserver started at http://127.8.208.65:42513/ using document root <none> and password file <none>
I20260929 03:02:48.448845 26239 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:48.448941 26239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:48.449244 26239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:48.451982 26239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "2e3b793335824ec796257a536463eda2"
format_stamp: "Formatted at 2026-09-29 03:02:48 on dist-test-slave-m1rv"
server_key: "04c98e58fafe96bc7c30f8617503dee4"
server_key_iv: "97a6e4036de1dab96050b915154dce9e"
server_key_version: "encryptionkey@0"
I20260929 03:02:48.452690 26239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "2e3b793335824ec796257a536463eda2"
format_stamp: "Formatted at 2026-09-29 03:02:48 on dist-test-slave-m1rv"
server_key: "04c98e58fafe96bc7c30f8617503dee4"
server_key_iv: "97a6e4036de1dab96050b915154dce9e"
server_key_version: "encryptionkey@0"
I20260929 03:02:48.457849 26239 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260929 03:02:48.461548 26254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:48.462829 26239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:02:48.462962 26239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "2e3b793335824ec796257a536463eda2"
format_stamp: "Formatted at 2026-09-29 03:02:48 on dist-test-slave-m1rv"
server_key: "04c98e58fafe96bc7c30f8617503dee4"
server_key_iv: "97a6e4036de1dab96050b915154dce9e"
server_key_version: "encryptionkey@0"
I20260929 03:02:48.463140 26239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:48.479974 26239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:48.483729 26239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:48.483958 26239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:48.484884 26239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:48.486284 26239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:48.486385 26239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:48.486532 26239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:48.486603 26239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:48.518931 26239 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:43353
I20260929 03:02:48.518980 26366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:43353 every 8 connection(s)
I20260929 03:02:48.520251 26239 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:02:48.526021 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 26239
I20260929 03:02:48.526242 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 03:02:48.527063 9025 external_mini_cluster.cc:1428] Setting key 2ee3a472d0d4bc96561ad24b5f29f4ce
Sep 29 03:02:48 dist-test-slave-m1rv krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650968, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:02:48.542644 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:48.522718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:38647 (local address 127.8.208.126:46543)
0929 03:02:48.523227 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:48.523232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:48.524039 (+ 807us) server_negotiation.cc:407] Connection header received
0929 03:02:48.526353 (+ 2314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:48.526360 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:48.526442 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:48.526546 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:48.528473 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:48.529356 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:48.531909 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:48.532301 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:48.532399 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:48.536641 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:48.536646 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:48.536650 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:48.536682 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:48.539317 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:48.540283 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:48.540288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:48.540290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:48.540406 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:48.540894 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:48.540897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:48.540899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:48.541146 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:48.541313 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:48.542198 (+ 885us) server_negotiation.cc:299] Negotiation successful
0929 03:02:48.542365 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":263,"threads_started":1}
I20260929 03:02:48.544160 26370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:48.523181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46543 (local address 127.8.208.65:38647)
0929 03:02:48.523870 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:48.523911 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:48.526014 (+ 2103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:48.526842 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:48.526855 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:48.527637 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:48.528311 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:48.528336 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:48.529527 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:48.529532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:48.531759 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:48.531777 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:48.532534 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:48.532544 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:48.532867 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:48.533620 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:02:48.533655 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:48.536383 (+ 2728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:48.539458 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:48.539469 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:48.539489 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:48.540148 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:48.540511 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:48.540518 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:48.540532 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:48.540787 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:48.541314 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:48.541325 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:48.541805 (+ 480us) client_negotiation.cc:769] Sending connection context
0929 03:02:48.542120 (+ 315us) client_negotiation.cc:240] Negotiation successful
0929 03:02:48.542699 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":263,"threads_started":1}
I20260929 03:02:48.545807 26367 heartbeater.cc:347] Connected to a master server at 127.8.208.126:46543
I20260929 03:02:48.546114 26367 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:48.546830 26367 heartbeater.cc:511] Master 127.8.208.126:46543 requested a full tablet report, sending...
I20260929 03:02:48.549526 26178 ts_manager.cc:194] Registered new tserver with Master: 2e3b793335824ec796257a536463eda2 (127.8.208.65:43353)
I20260929 03:02:48.551548 26178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:38647
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 03:02:48.585887 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46543
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:43135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:02:48.902057 26373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:48.902326 26373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:48.902498 26373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:48.909598 26373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:48.909700 26373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:48.909915 26373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:02:48.919570 26373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:43135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46543
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:48.920830 26373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:48.922137 26373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:48.934840 26378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:48.934875 26379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:48.935000 26381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:48.935784 26373 server_base.cc:1034] running on GCE node
I20260929 03:02:48.936384 26373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:48.937482 26373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:48.938700 26373 hybrid_clock.cc:648] HybridClock initialized: now 1790650968938664 us; error 38 us; skew 500 ppm
I20260929 03:02:48.939026 26373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:48 dist-test-slave-m1rv krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650968, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:48.942240 26373 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 03:02:48.943665 26373 webserver.cc:460] Webserver started at http://127.8.208.66:45579/ using document root <none> and password file <none>
I20260929 03:02:48.944339 26373 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:48.944480 26373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:48.944787 26373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:48.947933 26373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "448dc88248e447998f73d4fd51e18a3c"
format_stamp: "Formatted at 2026-09-29 03:02:48 on dist-test-slave-m1rv"
server_key: "0e5f4a254159313f0eb4180a0f41aa91"
server_key_iv: "22825408bb0cb20e6f97198d1c263e7b"
server_key_version: "encryptionkey@0"
I20260929 03:02:48.948694 26373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "448dc88248e447998f73d4fd51e18a3c"
format_stamp: "Formatted at 2026-09-29 03:02:48 on dist-test-slave-m1rv"
server_key: "0e5f4a254159313f0eb4180a0f41aa91"
server_key_iv: "22825408bb0cb20e6f97198d1c263e7b"
server_key_version: "encryptionkey@0"
I20260929 03:02:48.953648 26373 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 03:02:48.956923 26388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:48.958302 26373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:02:48.958457 26373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "448dc88248e447998f73d4fd51e18a3c"
format_stamp: "Formatted at 2026-09-29 03:02:48 on dist-test-slave-m1rv"
server_key: "0e5f4a254159313f0eb4180a0f41aa91"
server_key_iv: "22825408bb0cb20e6f97198d1c263e7b"
server_key_version: "encryptionkey@0"
I20260929 03:02:48.958644 26373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:48.980976 26373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:48.985286 26373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:48.985571 26373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:48.986547 26373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:48.987860 26373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:48.987937 26373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:48.988107 26373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:48.988173 26373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:49.021823 26373 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:41867
I20260929 03:02:49.021962 26500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:41867 every 8 connection(s)
I20260929 03:02:49.023161 26373 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:02:49.028061 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 26373
I20260929 03:02:49.028297 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 03:02:49.029018 9025 external_mini_cluster.cc:1428] Setting key 2475600f6b731b15249e3220256b80bb
Sep 29 03:02:49 dist-test-slave-m1rv krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650968, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:02:49.050129 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.026225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:48991 (local address 127.8.208.126:46543)
0929 03:02:49.026439 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:49.026447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:49.027770 (+ 1323us) server_negotiation.cc:407] Connection header received
0929 03:02:49.030689 (+ 2919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:49.030695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:49.030769 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:49.030878 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:49.033127 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.033874 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.036709 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.037123 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.037299 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.041947 (+ 4648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:49.041957 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:49.041967 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:49.042032 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:49.046460 (+ 4428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:49.047189 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:49.047199 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:49.047206 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:49.047392 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:49.047890 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:49.047899 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:49.047906 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:49.048364 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:49.048536 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:49.049252 (+ 716us) server_negotiation.cc:299] Negotiation successful
0929 03:02:49.049559 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 03:02:49.050616 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.026678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46543 (local address 127.8.208.66:48991)
0929 03:02:49.027569 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:49.027624 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:49.030415 (+ 2791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:49.031178 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:49.031188 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:49.032008 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:49.032924 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.032965 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.034034 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.034044 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.036530 (+ 2486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.036552 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.037435 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.037448 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.037841 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.038647 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:02:49.038688 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:49.041645 (+ 2957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:49.046612 (+ 4967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:49.046618 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:49.046632 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:49.047055 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:49.047571 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:49.047574 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:49.047576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:49.047754 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:49.048497 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:49.048503 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:49.048828 (+ 325us) client_negotiation.cc:769] Sending connection context
0929 03:02:49.049065 (+ 237us) client_negotiation.cc:240] Negotiation successful
0929 03:02:49.049505 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":372,"threads_started":1}
I20260929 03:02:49.052196 26501 heartbeater.cc:347] Connected to a master server at 127.8.208.126:46543
I20260929 03:02:49.052583 26501 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:49.053638 26501 heartbeater.cc:511] Master 127.8.208.126:46543 requested a full tablet report, sending...
I20260929 03:02:49.055325 26178 ts_manager.cc:194] Registered new tserver with Master: 448dc88248e447998f73d4fd51e18a3c (127.8.208.66:41867)
I20260929 03:02:49.056154 26178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:48991
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 03:02:49.092444 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46543
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:43135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:02:49.420557 26506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:49.420832 26506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:49.420935 26506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:49.428279 26506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:49.428381 26506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:49.428495 26506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:02:49.438493 26506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:43135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46543
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:49.439674 26506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:49.440865 26506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:49.454154 26511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:49.454169 26514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:49.454201 26512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:49.455018 26506 server_base.cc:1034] running on GCE node
I20260929 03:02:49.455641 26506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:49.456691 26506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:49.457979 26506 hybrid_clock.cc:648] HybridClock initialized: now 1790650969457938 us; error 53 us; skew 500 ppm
I20260929 03:02:49.458312 26506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:49 dist-test-slave-m1rv krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650969, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:49.461753 26506 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 03:02:49.463358 26506 webserver.cc:460] Webserver started at http://127.8.208.67:45873/ using document root <none> and password file <none>
I20260929 03:02:49.464012 26506 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:49.464146 26506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:49.464440 26506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:49.467437 26506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "a7d628c9f2bc45d887d464b20776c632"
format_stamp: "Formatted at 2026-09-29 03:02:49 on dist-test-slave-m1rv"
server_key: "f30d0f34ebb559d3e8fc4378b70bfa39"
server_key_iv: "6398026d7a25cd80e34eecf65db75d37"
server_key_version: "encryptionkey@0"
I20260929 03:02:49.468189 26506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "a7d628c9f2bc45d887d464b20776c632"
format_stamp: "Formatted at 2026-09-29 03:02:49 on dist-test-slave-m1rv"
server_key: "f30d0f34ebb559d3e8fc4378b70bfa39"
server_key_iv: "6398026d7a25cd80e34eecf65db75d37"
server_key_version: "encryptionkey@0"
I20260929 03:02:49.473404 26506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 03:02:49.477329 26521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:49.478638 26506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:02:49.478811 26506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "a7d628c9f2bc45d887d464b20776c632"
format_stamp: "Formatted at 2026-09-29 03:02:49 on dist-test-slave-m1rv"
server_key: "f30d0f34ebb559d3e8fc4378b70bfa39"
server_key_iv: "6398026d7a25cd80e34eecf65db75d37"
server_key_version: "encryptionkey@0"
I20260929 03:02:49.479033 26506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:49.494894 26506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:49.498863 26506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:49.499166 26506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:49.500097 26506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:49.501451 26506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:49.501555 26506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:49.501708 26506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:49.501777 26506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:49.535346 26506 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:36587
I20260929 03:02:49.535430 26633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:36587 every 8 connection(s)
I20260929 03:02:49.536635 26506 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:02:49.546304 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 26506
I20260929 03:02:49.546494 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 03:02:49.547057 9025 external_mini_cluster.cc:1428] Setting key d927251ec19f73f9c2d669529d21d013
Sep 29 03:02:49 dist-test-slave-m1rv krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650969, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:02:49.554675 26367 heartbeater.cc:503] Master 127.8.208.126:46543 was elected leader, sending a full tablet report...
I20260929 03:02:49.558069 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.539173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:35207 (local address 127.8.208.126:46543)
0929 03:02:49.539343 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:49.539347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:49.540552 (+ 1205us) server_negotiation.cc:407] Connection header received
0929 03:02:49.542409 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:49.542414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:49.542484 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:49.542566 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:49.543939 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.544753 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.546553 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.546967 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.547081 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.550782 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:49.550786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:49.550789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:49.550820 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:49.554513 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:49.555376 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:49.555382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:49.555385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:49.555552 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:49.556102 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:49.556106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:49.556109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:49.556507 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:49.556662 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:49.557497 (+ 835us) server_negotiation.cc:299] Negotiation successful
0929 03:02:49.557709 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 03:02:49.558988 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.539613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46543 (local address 127.8.208.67:35207)
0929 03:02:49.540334 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:49.540371 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:49.542120 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:49.542760 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:49.542768 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:49.543268 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:49.543798 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.543814 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.544900 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.544905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.546381 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.546422 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.547202 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.547211 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.547527 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.548238 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:02:49.548259 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:49.550606 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:49.554683 (+ 4077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:49.554692 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:49.554707 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:49.555243 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:49.555694 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:49.555698 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:49.555701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:49.555891 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:49.556634 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:49.556642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:49.557065 (+ 423us) client_negotiation.cc:769] Sending connection context
0929 03:02:49.557395 (+ 330us) client_negotiation.cc:240] Negotiation successful
0929 03:02:49.557928 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":306,"threads_started":1}
I20260929 03:02:49.560437 26634 heartbeater.cc:347] Connected to a master server at 127.8.208.126:46543
I20260929 03:02:49.560746 26634 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:49.561520 26634 heartbeater.cc:511] Master 127.8.208.126:46543 requested a full tablet report, sending...
I20260929 03:02:49.562876 26178 ts_manager.cc:194] Registered new tserver with Master: a7d628c9f2bc45d887d464b20776c632 (127.8.208.67:36587)
I20260929 03:02:49.563735 26178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:35207
I20260929 03:02:49.574313 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:02:49.592113 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.578756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46730 (local address 127.8.208.126:46543)
0929 03:02:49.578926 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:49.578931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:49.579433 (+ 502us) server_negotiation.cc:407] Connection header received
0929 03:02:49.579570 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:49.579574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:49.579637 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:49.579726 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:49.580993 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.581972 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.583342 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.583845 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.583978 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.586417 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:49.586422 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:49.586426 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:49.586469 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:49.589465 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:49.590139 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:49.590144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:49.590147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:49.590282 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:49.590687 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:49.590691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:49.590694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:49.591104 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:49.591311 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:49.591631 (+ 320us) server_negotiation.cc:299] Negotiation successful
0929 03:02:49.591813 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:02:49.596453 26178 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:02:49.603775 26178 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:02:49.626892 26646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.615727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:36587 (local address 127.0.0.1:49032)
0929 03:02:49.617125 (+ 1398us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:49.617204 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:49.617355 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:49.618377 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:49.618385 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:49.618431 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:49.619544 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.619559 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.621542 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.621547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.623992 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.624007 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.625807 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.625812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.626015 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.626071 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 03:02:49.626241 (+ 170us) client_negotiation.cc:240] Negotiation successful
0929 03:02:49.626422 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"mutex_wait_us":76,"thread_start_us":958,"threads_started":1}
I20260929 03:02:49.627065 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.617528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:41867 (local address 127.0.0.1:33392)
0929 03:02:49.618060 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:49.618081 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:49.618219 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:49.620808 (+ 2589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:49.620815 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:49.620839 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:49.621110 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.621117 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.622579 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.622585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.624629 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.624639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.626232 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.626239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.626412 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.626439 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 03:02:49.626525 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 03:02:49.626651 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":25,"thread_start_us":238,"threads_started":1}
I20260929 03:02:49.627962 26650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.618076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:43353 (local address 127.0.0.1:34084)
0929 03:02:49.618858 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:49.618878 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:49.619006 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:49.620271 (+ 1265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:49.620275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:49.620299 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:49.620670 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.620679 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.623033 (+ 2354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.623038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.625317 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.625331 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.627160 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.627169 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.627342 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.627373 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 03:02:49.627439 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 03:02:49.627561 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":1,"thread_start_us":802,"threads_started":1}
I20260929 03:02:49.627975 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.615990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49032 (local address 127.8.208.67:36587)
0929 03:02:49.617344 (+ 1354us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:49.617355 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:49.617390 (+ 35us) server_negotiation.cc:407] Connection header received
0929 03:02:49.617581 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:49.617592 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:49.617857 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:49.618069 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:49.619764 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.621400 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.624171 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.625627 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.625768 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.626002 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:49.626441 (+ 439us) server_negotiation.cc:299] Negotiation successful
0929 03:02:49.627628 (+ 1187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1193,"thread_start_us":1009,"threads_started":1}
I20260929 03:02:49.627990 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.617760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.8.208.66:41867)
0929 03:02:49.620300 (+ 2540us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:49.620308 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:49.620335 (+ 27us) server_negotiation.cc:407] Connection header received
0929 03:02:49.620420 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:49.620428 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:49.620628 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:49.620808 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:49.621286 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.622404 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.624771 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.626008 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.626152 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.626330 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:49.627277 (+ 947us) server_negotiation.cc:299] Negotiation successful
0929 03:02:49.627635 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2380,"thread_start_us":2244,"threads_started":1}
I20260929 03:02:49.629071 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.618672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34084 (local address 127.8.208.65:43353)
0929 03:02:49.619669 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:49.619681 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:49.619715 (+ 34us) server_negotiation.cc:407] Connection header received
0929 03:02:49.619826 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:49.619836 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:49.620097 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:49.620286 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:49.621323 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.622865 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.625478 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.626892 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.628320 (+ 1428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.628525 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:49.628588 (+ 63us) server_negotiation.cc:299] Negotiation successful
0929 03:02:49.628743 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":587,"threads_started":1}
I20260929 03:02:49.631284 26569 tablet_service.cc:1467] Processing CreateTablet for tablet bba142142ef94cb18402913ad6d99e0a (DEFAULT_TABLE table=test-table [id=b173b08367a3445d80ce7228d27f4c93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:02:49.631506 26436 tablet_service.cc:1467] Processing CreateTablet for tablet bba142142ef94cb18402913ad6d99e0a (DEFAULT_TABLE table=test-table [id=b173b08367a3445d80ce7228d27f4c93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:02:49.632296 26302 tablet_service.cc:1467] Processing CreateTablet for tablet bba142142ef94cb18402913ad6d99e0a (DEFAULT_TABLE table=test-table [id=b173b08367a3445d80ce7228d27f4c93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:02:49.632840 26436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba142142ef94cb18402913ad6d99e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:49.632820 26569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba142142ef94cb18402913ad6d99e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:49.633594 26302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba142142ef94cb18402913ad6d99e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:49.644065 26652 tablet_bootstrap.cc:492] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: Bootstrap starting.
I20260929 03:02:49.647629 26653 tablet_bootstrap.cc:492] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2: Bootstrap starting.
I20260929 03:02:49.648533 26652 tablet_bootstrap.cc:654] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:49.649892 26652 log.cc:824] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:49.650120 26654 tablet_bootstrap.cc:492] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c: Bootstrap starting.
I20260929 03:02:49.651921 26653 tablet_bootstrap.cc:654] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:49.652900 26654 tablet_bootstrap.cc:654] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:49.653673 26653 log.cc:824] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:49.654147 26654 log.cc:824] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:49.655761 26652 tablet_bootstrap.cc:492] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: No bootstrap required, opened a new log
I20260929 03:02:49.656047 26652 ts_tablet_manager.cc:1397] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260929 03:02:49.656862 26654 tablet_bootstrap.cc:492] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c: No bootstrap required, opened a new log
I20260929 03:02:49.657070 26653 tablet_bootstrap.cc:492] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2: No bootstrap required, opened a new log
I20260929 03:02:49.657119 26654 ts_tablet_manager.cc:1397] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260929 03:02:49.657352 26653 ts_tablet_manager.cc:1397] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260929 03:02:49.665499 26654 raft_consensus.cc:348] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7d628c9f2bc45d887d464b20776c632" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36587 } } peers { permanent_uuid: "448dc88248e447998f73d4fd51e18a3c" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41867 } } peers { permanent_uuid: "2e3b793335824ec796257a536463eda2" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 43353 } }
I20260929 03:02:49.665881 26654 raft_consensus.cc:374] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:49.666039 26654 raft_consensus.cc:729] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 448dc88248e447998f73d4fd51e18a3c, State: Initialized, Role: FOLLOWER
I20260929 03:02:49.666730 26654 consensus_queue.cc:260] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7d628c9f2bc45d887d464b20776c632" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36587 } } peers { permanent_uuid: "448dc88248e447998f73d4fd51e18a3c" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41867 } } peers { permanent_uuid: "2e3b793335824ec796257a536463eda2" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 43353 } }
I20260929 03:02:49.666899 26652 raft_consensus.cc:348] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7d628c9f2bc45d887d464b20776c632" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36587 } } peers { permanent_uuid: "448dc88248e447998f73d4fd51e18a3c" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41867 } } peers { permanent_uuid: "2e3b793335824ec796257a536463eda2" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 43353 } }
I20260929 03:02:49.667384 26652 raft_consensus.cc:374] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:49.667511 26652 raft_consensus.cc:729] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d628c9f2bc45d887d464b20776c632, State: Initialized, Role: FOLLOWER
I20260929 03:02:49.668144 26501 heartbeater.cc:503] Master 127.8.208.126:46543 was elected leader, sending a full tablet report...
I20260929 03:02:49.668111 26652 consensus_queue.cc:260] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7d628c9f2bc45d887d464b20776c632" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36587 } } peers { permanent_uuid: "448dc88248e447998f73d4fd51e18a3c" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41867 } } peers { permanent_uuid: "2e3b793335824ec796257a536463eda2" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 43353 } }
I20260929 03:02:49.668071 26653 raft_consensus.cc:348] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7d628c9f2bc45d887d464b20776c632" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36587 } } peers { permanent_uuid: "448dc88248e447998f73d4fd51e18a3c" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41867 } } peers { permanent_uuid: "2e3b793335824ec796257a536463eda2" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 43353 } }
I20260929 03:02:49.668465 26653 raft_consensus.cc:374] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:49.668587 26654 ts_tablet_manager.cc:1428] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260929 03:02:49.668569 26653 raft_consensus.cc:729] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e3b793335824ec796257a536463eda2, State: Initialized, Role: FOLLOWER
I20260929 03:02:49.669203 26653 consensus_queue.cc:260] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7d628c9f2bc45d887d464b20776c632" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36587 } } peers { permanent_uuid: "448dc88248e447998f73d4fd51e18a3c" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41867 } } peers { permanent_uuid: "2e3b793335824ec796257a536463eda2" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 43353 } }
I20260929 03:02:49.669557 26634 heartbeater.cc:503] Master 127.8.208.126:46543 was elected leader, sending a full tablet report...
I20260929 03:02:49.670382 26652 ts_tablet_manager.cc:1428] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260929 03:02:49.673763 26653 ts_tablet_manager.cc:1428] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2: Time spent starting tablet: real 0.016s user 0.009s sys 0.005s
W20260929 03:02:49.772845 26368 tablet.cc:2369] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:02:49.775106 26502 tablet.cc:2369] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:02:49.788236 26635 tablet.cc:2369] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:02:49.930815 26660 raft_consensus.cc:482] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:02:49.931128 26660 raft_consensus.cc:504] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7d628c9f2bc45d887d464b20776c632" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36587 } } peers { permanent_uuid: "448dc88248e447998f73d4fd51e18a3c" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41867 } } peers { permanent_uuid: "2e3b793335824ec796257a536463eda2" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 43353 } }
I20260929 03:02:49.932727 26660 leader_election.cc:288] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7d628c9f2bc45d887d464b20776c632 (127.8.208.67:36587), 448dc88248e447998f73d4fd51e18a3c (127.8.208.66:41867)
I20260929 03:02:49.941154 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.933141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:44449 (local address 127.8.208.67:36587)
0929 03:02:49.933339 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:49.933344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:49.934029 (+ 685us) server_negotiation.cc:407] Connection header received
0929 03:02:49.934294 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:49.934300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:49.934378 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:49.934447 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:49.935364 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.936807 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.938857 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.939991 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.940158 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.940257 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:49.940734 (+ 477us) server_negotiation.cc:299] Negotiation successful
0929 03:02:49.940869 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 03:02:49.941375 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.933250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:36587 (local address 127.8.208.65:44449)
0929 03:02:49.933909 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:49.933924 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:49.934113 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:49.934718 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:49.934726 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:49.934764 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:49.935209 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.935221 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.936941 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.936944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.938699 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.938710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.940247 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.940255 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.940526 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.940582 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 03:02:49.940694 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 03:02:49.940879 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":391,"threads_started":1}
I20260929 03:02:49.942613 26589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba142142ef94cb18402913ad6d99e0a" candidate_uuid: "2e3b793335824ec796257a536463eda2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d628c9f2bc45d887d464b20776c632" is_pre_election: true
I20260929 03:02:49.943241 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.934324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:41809 (local address 127.8.208.66:41867)
0929 03:02:49.934525 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:49.934531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:49.935201 (+ 670us) server_negotiation.cc:407] Connection header received
0929 03:02:49.937035 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:49.937043 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:49.937142 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:49.937257 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:49.937955 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.939199 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.941346 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.942356 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.942539 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.942612 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:49.942861 (+ 249us) server_negotiation.cc:299] Negotiation successful
0929 03:02:49.942988 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 03:02:49.943346 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.934199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:41867 (local address 127.8.208.65:41809)
0929 03:02:49.935065 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:49.935088 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:49.935221 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:49.937303 (+ 2082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:49.937308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:49.937334 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:49.937778 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.937795 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.939350 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.939356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.941134 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:49.941151 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.942557 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.942563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:49.942712 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.942735 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 03:02:49.942836 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 03:02:49.942961 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":1,"thread_start_us":477,"threads_started":1}
I20260929 03:02:49.943286 26589 raft_consensus.cc:2454] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e3b793335824ec796257a536463eda2 in term 0.
I20260929 03:02:49.944195 26256 leader_election.cc:302] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e3b793335824ec796257a536463eda2, a7d628c9f2bc45d887d464b20776c632; no voters:
I20260929 03:02:49.944450 26456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba142142ef94cb18402913ad6d99e0a" candidate_uuid: "2e3b793335824ec796257a536463eda2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448dc88248e447998f73d4fd51e18a3c" is_pre_election: true
I20260929 03:02:49.944716 26660 raft_consensus.cc:2781] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:02:49.944886 26660 raft_consensus.cc:482] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:02:49.944984 26660 raft_consensus.cc:3037] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:49.945055 26456 raft_consensus.cc:2454] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e3b793335824ec796257a536463eda2 in term 0.
I20260929 03:02:49.947404 26660 raft_consensus.cc:504] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7d628c9f2bc45d887d464b20776c632" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36587 } } peers { permanent_uuid: "448dc88248e447998f73d4fd51e18a3c" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41867 } } peers { permanent_uuid: "2e3b793335824ec796257a536463eda2" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 43353 } }
I20260929 03:02:49.948024 26660 leader_election.cc:288] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [CANDIDATE]: Term 1 election: Requested vote from peers a7d628c9f2bc45d887d464b20776c632 (127.8.208.67:36587), 448dc88248e447998f73d4fd51e18a3c (127.8.208.66:41867)
I20260929 03:02:49.948525 26589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba142142ef94cb18402913ad6d99e0a" candidate_uuid: "2e3b793335824ec796257a536463eda2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d628c9f2bc45d887d464b20776c632"
I20260929 03:02:49.948587 26456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba142142ef94cb18402913ad6d99e0a" candidate_uuid: "2e3b793335824ec796257a536463eda2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "448dc88248e447998f73d4fd51e18a3c"
I20260929 03:02:49.948733 26589 raft_consensus.cc:3037] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:49.948819 26456 raft_consensus.cc:3037] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:49.951190 26589 raft_consensus.cc:2454] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e3b793335824ec796257a536463eda2 in term 1.
I20260929 03:02:49.951596 26256 leader_election.cc:302] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e3b793335824ec796257a536463eda2, a7d628c9f2bc45d887d464b20776c632; no voters:
I20260929 03:02:49.951887 26660 raft_consensus.cc:2781] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:02:49.952415 26660 raft_consensus.cc:686] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 1 LEADER]: Becoming Leader. State: Replica: 2e3b793335824ec796257a536463eda2, State: Running, Role: LEADER
I20260929 03:02:49.952651 26456 raft_consensus.cc:2454] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e3b793335824ec796257a536463eda2 in term 1.
I20260929 03:02:49.952988 26660 consensus_queue.cc:237] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7d628c9f2bc45d887d464b20776c632" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36587 } } peers { permanent_uuid: "448dc88248e447998f73d4fd51e18a3c" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41867 } } peers { permanent_uuid: "2e3b793335824ec796257a536463eda2" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 43353 } }
I20260929 03:02:49.958441 26177 catalog_manager.cc:5495] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e3b793335824ec796257a536463eda2 (127.8.208.65). New cstate: current_term: 1 leader_uuid: "2e3b793335824ec796257a536463eda2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7d628c9f2bc45d887d464b20776c632" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "448dc88248e447998f73d4fd51e18a3c" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e3b793335824ec796257a536463eda2" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 43353 } health_report { overall_health: HEALTHY } } }
I20260929 03:02:49.992271 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:49.986812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34088 (local address 127.8.208.65:43353)
0929 03:02:49.987016 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:49.987033 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:49.987173 (+ 140us) server_negotiation.cc:407] Connection header received
0929 03:02:49.987407 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:49.987411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:49.987504 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:49.987651 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:02:49.988030 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.988712 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.990147 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:49.990543 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:49.990667 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:49.991186 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:02:49.991691 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:02:49.991848 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:49.991914 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 03:02:49.991994 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 03:02:50.000399 26589 raft_consensus.cc:1261] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632 [term 1 FOLLOWER]: Refusing update from remote peer 2e3b793335824ec796257a536463eda2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:02:50.000452 26456 raft_consensus.cc:1261] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c [term 1 FOLLOWER]: Refusing update from remote peer 2e3b793335824ec796257a536463eda2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:02:50.001469 26660 consensus_queue.cc:1035] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d628c9f2bc45d887d464b20776c632" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 36587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:02:50.001852 26663 consensus_queue.cc:1035] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "448dc88248e447998f73d4fd51e18a3c" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:02:50.056116 26177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:02:50.056486 26177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:02:50.061645 26177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d211c56def714fd0a6586a51227e90d4: Sending DeleteTablet for 3 replicas of tablet bba142142ef94cb18402913ad6d99e0a
I20260929 03:02:50.063236 26302 tablet_service.cc:1514] Processing DeleteTablet for tablet bba142142ef94cb18402913ad6d99e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:02:50 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:34084
I20260929 03:02:50.064190 26674 tablet_replica.cc:330] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2: stopping tablet replica
I20260929 03:02:50.064622 26436 tablet_service.cc:1514] Processing DeleteTablet for tablet bba142142ef94cb18402913ad6d99e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:02:50 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:33392
I20260929 03:02:50.064723 26674 raft_consensus.cc:2229] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:02:50.065413 26674 raft_consensus.cc:2258] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:02:50.066589 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 26239
I20260929 03:02:50.067423 26674 ts_tablet_manager.cc:1905] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:02:50.071869 26569 tablet_service.cc:1514] Processing DeleteTablet for tablet bba142142ef94cb18402913ad6d99e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:02:50 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:49032
I20260929 03:02:50.083050 26677 tablet_replica.cc:330] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: stopping tablet replica
I20260929 03:02:50.083662 26677 raft_consensus.cc:2229] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:02:50.084165 26677 pending_rounds.cc:62] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: Trying to abort 2 pending ops.
I20260929 03:02:50.084316 26677 pending_rounds.cc:68] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7334506373111640064 op_type: WRITE_OP write_request { tablet_id: "bba142142ef94cb18402913ad6d99e0a" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\205\327\354\325\006\"8\n\ntest-admin\022*\n b173b08367a3445d80ce7228d27f4c93\020\001\030\001 \001(\001" signature: "<redacted>""\316RB\2573\034\306\261C\252Q\364\007\241O\323^d\246\372\350;Z)\022\201\371\312\305\"u\2532\257\302C\325\230m\371\024\n\214\016\341}\001\225F\304\000\177\206\275j\207!\317f\n\241\313\341\337" signing_key_seq_num: 0 } } request_id { client_id: "e471ba936da34d6d9a41f29e3ebac664" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260929 03:02:50.086139 26677 pending_rounds.cc:68] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334506372931235840 op_type: NO_OP noop_request { }
I20260929 03:02:50.086366 26677 raft_consensus.cc:2866] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260929 03:02:50.086585 26677 raft_consensus.cc:2258] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:02:50.093940 26674 ts_tablet_manager.cc:1918] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:02:50.095027 26674 log.cc:1196] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/bba142142ef94cb18402913ad6d99e0a
I20260929 03:02:50.095966 26674 ts_tablet_manager.cc:1939] T bba142142ef94cb18402913ad6d99e0a P 2e3b793335824ec796257a536463eda2: Deleting consensus metadata
I20260929 03:02:50.098861 26163 catalog_manager.cc:4841] TS 2e3b793335824ec796257a536463eda2 (127.8.208.65:43353): tablet bba142142ef94cb18402913ad6d99e0a (table test-table [id=b173b08367a3445d80ce7228d27f4c93]) successfully deleted
I20260929 03:02:50.100140 26675 tablet_replica.cc:330] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c: stopping tablet replica
I20260929 03:02:50.100797 26675 raft_consensus.cc:2229] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:02:50.103132 26677 ts_tablet_manager.cc:1905] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:02:50.103771 26675 raft_consensus.cc:2258] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:02:50.106257 26675 ts_tablet_manager.cc:1905] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:02:50.116448 26677 ts_tablet_manager.cc:1918] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:02:50.118132 26675 ts_tablet_manager.cc:1918] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:02:50.125334 26677 log.cc:1196] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/bba142142ef94cb18402913ad6d99e0a
I20260929 03:02:50.126029 26677 ts_tablet_manager.cc:1939] T bba142142ef94cb18402913ad6d99e0a P a7d628c9f2bc45d887d464b20776c632: Deleting consensus metadata
Sep 29 03:02:50 dist-test-slave-m1rv krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 03:02:50.128829 26163 catalog_manager.cc:4841] TS a7d628c9f2bc45d887d464b20776c632 (127.8.208.67:36587): tablet bba142142ef94cb18402913ad6d99e0a (table test-table [id=b173b08367a3445d80ce7228d27f4c93]) successfully deleted
I20260929 03:02:50.129552 26675 log.cc:1196] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/bba142142ef94cb18402913ad6d99e0a
I20260929 03:02:50.130203 26675 ts_tablet_manager.cc:1939] T bba142142ef94cb18402913ad6d99e0a P 448dc88248e447998f73d4fd51e18a3c: Deleting consensus metadata
I20260929 03:02:50.132268 26163 catalog_manager.cc:4841] TS 448dc88248e447998f73d4fd51e18a3c (127.8.208.66:41867): tablet bba142142ef94cb18402913ad6d99e0a (table test-table [id=b173b08367a3445d80ce7228d27f4c93]) successfully deleted
I20260929 03:02:50.139662 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:50.068477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.8.208.65:43353)
0929 03:02:50.068956 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:50.068962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:50.106997 (+ 38035us) server_negotiation.cc:407] Connection header received
0929 03:02:50.107320 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:50.107329 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:50.107430 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:50.107532 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:50.118447 (+ 10915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:50.119958 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:50.122601 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:50.123473 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:50.123590 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:50.130543 (+ 6953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:50.130566 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:50.130589 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:50.130644 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:50.135654 (+ 5010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:50.136729 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:50.136741 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:50.136747 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:50.136952 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:50.137503 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:50.137511 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:50.137516 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:50.138022 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:50.138362 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:50.138798 (+ 436us) server_negotiation.cc:299] Negotiation successful
0929 03:02:50.139138 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352}
I20260929 03:02:50.140506 26362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:50.280997 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 26239
I20260929 03:02:50.298851 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 26373
Sep 29 03:02:50 dist-test-slave-m1rv krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 03:02:50.313596 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:50.299629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33400 (local address 127.8.208.66:41867)
0929 03:02:50.299814 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:50.299820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:50.299910 (+ 90us) server_negotiation.cc:407] Connection header received
0929 03:02:50.300115 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:50.300120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:50.300189 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:50.300303 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:50.301643 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:50.302358 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:50.303593 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:50.303994 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:50.304131 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:50.308145 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:50.308157 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:50.308174 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:50.308203 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:50.311037 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:50.311632 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:50.311638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:50.311640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:50.311755 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:50.312149 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:50.312152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:50.312154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:50.312481 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:50.312696 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:50.312984 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 03:02:50.313137 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 03:02:50.314213 26496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:50.403062 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 26373
I20260929 03:02:50.420461 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 26506
Sep 29 03:02:50 dist-test-slave-m1rv krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 03:02:50.436466 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:50.421266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49042 (local address 127.8.208.67:36587)
0929 03:02:50.421423 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:50.421427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:50.421532 (+ 105us) server_negotiation.cc:407] Connection header received
0929 03:02:50.421684 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:50.421688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:50.421768 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:50.421851 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:50.423263 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:50.423993 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:50.425345 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:50.425733 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:50.425823 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:50.430940 (+ 5117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:50.430955 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:50.430971 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:50.431004 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:50.433939 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:50.434620 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:50.434627 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:50.434630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:50.434762 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:50.435113 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:50.435116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:50.435118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:50.435445 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:50.435681 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:50.435925 (+ 244us) server_negotiation.cc:299] Negotiation successful
0929 03:02:50.436091 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 03:02:50.437306 26629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:50.525141 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 26506
I20260929 03:02:50.541774 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 26146
I20260929 03:02:50.542870 26208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:50.652702 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 26146
2026-09-29T03:02:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4547 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T03:02:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:02:50Z Disabled control of system clock
I20260929 03:02:50.727105 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:40733
--webserver_interface=127.8.208.126
--webserver_port=0
--builtin_ntp_servers=127.8.208.84:39693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:40733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 03:02:51.059244 26685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:51.059525 26685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:51.059607 26685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:51.066785 26685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:02:51.067086 26685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:51.067217 26685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:51.067301 26685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:02:51.067354 26685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:02:51.077589 26685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:39693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:40733
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:40733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:51.078811 26685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:51.080552 26685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:51.092828 26691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:51.092914 26690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:51.092872 26693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:51.093750 26685 server_base.cc:1034] running on GCE node
I20260929 03:02:51.094678 26685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:51.096186 26685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:51.097415 26685 hybrid_clock.cc:648] HybridClock initialized: now 1790650971097366 us; error 59 us; skew 500 ppm
I20260929 03:02:51.097831 26685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 03:02:51.100155 26685 webserver.cc:460] Webserver started at http://127.8.208.126:39811/ using document root <none> and password file <none>
I20260929 03:02:51.100849 26685 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:51.100986 26685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:51.101348 26685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:51.104234 26685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "276a0cbc1b3e4b849a81324b81931434"
format_stamp: "Formatted at 2026-09-29 03:02:51 on dist-test-slave-m1rv"
server_key: "b4cb3debe748be671f8f38709733540e"
server_key_iv: "79c0652c099b62586a08563bffb7f09e"
server_key_version: "encryptionkey@0"
I20260929 03:02:51.105034 26685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "276a0cbc1b3e4b849a81324b81931434"
format_stamp: "Formatted at 2026-09-29 03:02:51 on dist-test-slave-m1rv"
server_key: "b4cb3debe748be671f8f38709733540e"
server_key_iv: "79c0652c099b62586a08563bffb7f09e"
server_key_version: "encryptionkey@0"
I20260929 03:02:51.110252 26685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 03:02:51.113723 26699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:51.115052 26685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 03:02:51.115229 26685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "276a0cbc1b3e4b849a81324b81931434"
format_stamp: "Formatted at 2026-09-29 03:02:51 on dist-test-slave-m1rv"
server_key: "b4cb3debe748be671f8f38709733540e"
server_key_iv: "79c0652c099b62586a08563bffb7f09e"
server_key_version: "encryptionkey@0"
I20260929 03:02:51.115442 26685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:51.139461 26685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:51.140326 26685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:51.140635 26685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:51.159081 26685 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:40733
I20260929 03:02:51.159101 26750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:40733 every 8 connection(s)
I20260929 03:02:51.160528 26685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:02:51.164443 26751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:51.169129 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 26685
I20260929 03:02:51.169364 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:02:51.169827 9025 external_mini_cluster.cc:1428] Setting key 9ee117c1cd62944d35a5125abd197e24
I20260929 03:02:51.175300 26751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434: Bootstrap starting.
I20260929 03:02:51.178421 26751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:51.179450 26751 log.cc:824] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:51.182530 26751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434: No bootstrap required, opened a new log
I20260929 03:02:51.183533 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:51.171878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43344 (local address 127.8.208.126:40733)
0929 03:02:51.172928 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:51.172939 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:51.172985 (+ 46us) server_negotiation.cc:407] Connection header received
0929 03:02:51.174735 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:51.174758 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:51.175214 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:51.175638 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:51.176189 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:51.177564 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:51.178975 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:51.179697 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:51.179889 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:51.181229 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:51.181240 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:51.181262 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:02:51.181806 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:51.181885 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:02:51.181906 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:51.182042 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:51.182250 (+ 208us) server_negotiation.cc:299] Negotiation successful
0929 03:02:51.182567 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":384,"threads_started":1}
I20260929 03:02:51.190554 26751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276a0cbc1b3e4b849a81324b81931434" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40733 } }
I20260929 03:02:51.190943 26751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:51.191058 26751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 276a0cbc1b3e4b849a81324b81931434, State: Initialized, Role: FOLLOWER
I20260929 03:02:51.191594 26751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276a0cbc1b3e4b849a81324b81931434" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40733 } }
I20260929 03:02:51.191835 26751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:02:51.191984 26751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:02:51.192152 26751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:51.194597 26751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276a0cbc1b3e4b849a81324b81931434" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40733 } }
I20260929 03:02:51.195116 26751 leader_election.cc:302] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 276a0cbc1b3e4b849a81324b81931434; no voters:
I20260929 03:02:51.195704 26751 leader_election.cc:288] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:02:51.195880 26756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:02:51.196666 26756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [term 1 LEADER]: Becoming Leader. State: Replica: 276a0cbc1b3e4b849a81324b81931434, State: Running, Role: LEADER
I20260929 03:02:51.197335 26756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276a0cbc1b3e4b849a81324b81931434" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40733 } }
I20260929 03:02:51.197985 26751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:02:51.200261 26758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 276a0cbc1b3e4b849a81324b81931434. Latest consensus state: current_term: 1 leader_uuid: "276a0cbc1b3e4b849a81324b81931434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276a0cbc1b3e4b849a81324b81931434" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40733 } } }
I20260929 03:02:51.200734 26758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [sys.catalog]: This master's current role is: LEADER
I20260929 03:02:51.200238 26757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "276a0cbc1b3e4b849a81324b81931434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276a0cbc1b3e4b849a81324b81931434" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 40733 } } }
I20260929 03:02:51.201135 26757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434 [sys.catalog]: This master's current role is: LEADER
I20260929 03:02:51.202247 26761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:02:51.207674 26761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:02:51.217414 26761 catalog_manager.cc:1347] Generated new cluster ID: 0ee075e57f8b4f26963808bd50ab332b
I20260929 03:02:51.217532 26761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:02:51.240207 26761 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:02:51.241549 26761 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:02:51.251001 26761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434: Generated new TSK 0
I20260929 03:02:51.251787 26761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:02:51.257838 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:40733
--builtin_ntp_servers=127.8.208.84:39693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 03:02:51.616392 26775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:51.616690 26775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:51.616847 26775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:51.624296 26775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:51.624395 26775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:51.624552 26775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:02:51.633512 26775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:39693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:40733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:51.634829 26775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:51.636641 26775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:51.649751 26780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:51.649720 26783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:51.649667 26781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:51.650712 26775 server_base.cc:1034] running on GCE node
I20260929 03:02:51.651326 26775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:51.652175 26775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:51.653362 26775 hybrid_clock.cc:648] HybridClock initialized: now 1790650971653290 us; error 62 us; skew 500 ppm
I20260929 03:02:51.653667 26775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 03:02:51.655861 26775 webserver.cc:460] Webserver started at http://127.8.208.65:34867/ using document root <none> and password file <none>
I20260929 03:02:51.656572 26775 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:51.656694 26775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:51.657007 26775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:51.659767 26775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "b1ec6127350d44d6af307b0fec0f7f14"
format_stamp: "Formatted at 2026-09-29 03:02:51 on dist-test-slave-m1rv"
server_key: "40ab54b4b0d92c88ca6086397831506f"
server_key_iv: "7776069763f0174d1e3a62d97002081d"
server_key_version: "encryptionkey@0"
I20260929 03:02:51.660480 26775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "b1ec6127350d44d6af307b0fec0f7f14"
format_stamp: "Formatted at 2026-09-29 03:02:51 on dist-test-slave-m1rv"
server_key: "40ab54b4b0d92c88ca6086397831506f"
server_key_iv: "7776069763f0174d1e3a62d97002081d"
server_key_version: "encryptionkey@0"
I20260929 03:02:51.665319 26775 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 03:02:51.668740 26789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:51.670153 26775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:02:51.670325 26775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "b1ec6127350d44d6af307b0fec0f7f14"
format_stamp: "Formatted at 2026-09-29 03:02:51 on dist-test-slave-m1rv"
server_key: "40ab54b4b0d92c88ca6086397831506f"
server_key_iv: "7776069763f0174d1e3a62d97002081d"
server_key_version: "encryptionkey@0"
I20260929 03:02:51.670542 26775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:51.690562 26775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:51.691393 26775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:51.691697 26775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:51.692700 26775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:51.694278 26775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:51.694392 26775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:51.694515 26775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:51.694600 26775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:51.727804 26775 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:38385
I20260929 03:02:51.727886 26901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:38385 every 8 connection(s)
I20260929 03:02:51.729079 26775 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:02:51.738662 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 26775
I20260929 03:02:51.738848 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 03:02:51.739280 9025 external_mini_cluster.cc:1428] Setting key 6a817e9e9af306a2e04aac13521b7a45
I20260929 03:02:51.741357 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:51.731561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:35721 (local address 127.8.208.126:40733)
0929 03:02:51.732045 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:51.732050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:51.733201 (+ 1151us) server_negotiation.cc:407] Connection header received
0929 03:02:51.734932 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:51.734940 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:51.735032 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:51.735150 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:51.736175 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:51.736912 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:51.738494 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:51.738964 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:51.739097 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:51.740475 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:51.740481 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:51.740486 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:02:51.740511 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:51.740559 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:02:51.740571 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:51.740665 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:51.740871 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 03:02:51.740966 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":244,"threads_started":1}
I20260929 03:02:51.742010 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:40733
--builtin_ntp_servers=127.8.208.84:39693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 03:02:51.742345 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:51.732148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:40733 (local address 127.8.208.65:35721)
0929 03:02:51.732998 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:51.733047 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:51.734665 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:51.735382 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:51.735390 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:51.735473 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:51.736003 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:51.736019 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:51.737075 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:51.737079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:51.738356 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:51.738368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:51.739183 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:51.739193 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:51.739475 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:51.740167 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:02:51.740170 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:51.740211 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:02:51.740287 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:51.740663 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:51.740676 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:51.740686 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 03:02:51.740858 (+ 172us) client_negotiation.cc:240] Negotiation successful
0929 03:02:51.741271 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":261,"threads_started":1}
I20260929 03:02:51.743814 26902 heartbeater.cc:347] Connected to a master server at 127.8.208.126:40733
I20260929 03:02:51.744113 26902 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:51.744812 26902 heartbeater.cc:511] Master 127.8.208.126:40733 requested a full tablet report, sending...
I20260929 03:02:51.747045 26716 ts_manager.cc:194] Registered new tserver with Master: b1ec6127350d44d6af307b0fec0f7f14 (127.8.208.65:38385)
I20260929 03:02:51.749307 26716 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.208.65:35721
W20260929 03:02:52.080756 26906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:52.081089 26906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:52.081280 26906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:52.089886 26906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:52.090027 26906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:52.090212 26906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:02:52.100703 26906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:39693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:40733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:52.102115 26906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:52.104050 26906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:52.118444 26914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:52.118489 26912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:52.118523 26911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:52.119289 26906 server_base.cc:1034] running on GCE node
I20260929 03:02:52.119932 26906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:52.120913 26906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:52.122126 26906 hybrid_clock.cc:648] HybridClock initialized: now 1790650972122053 us; error 69 us; skew 500 ppm
I20260929 03:02:52.122416 26906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 03:02:52.124571 26906 webserver.cc:460] Webserver started at http://127.8.208.66:43761/ using document root <none> and password file <none>
I20260929 03:02:52.125272 26906 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:52.125393 26906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:52.125726 26906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:52.128362 26906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "4828db66bbd44453a01e1e0c8c4d935b"
format_stamp: "Formatted at 2026-09-29 03:02:52 on dist-test-slave-m1rv"
server_key: "a9a03ea4cdceb033622097f27d146ece"
server_key_iv: "66c1ba2e7bbfd122116b078744dd09d9"
server_key_version: "encryptionkey@0"
I20260929 03:02:52.129110 26906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "4828db66bbd44453a01e1e0c8c4d935b"
format_stamp: "Formatted at 2026-09-29 03:02:52 on dist-test-slave-m1rv"
server_key: "a9a03ea4cdceb033622097f27d146ece"
server_key_iv: "66c1ba2e7bbfd122116b078744dd09d9"
server_key_version: "encryptionkey@0"
I20260929 03:02:52.134117 26906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260929 03:02:52.137403 26920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:52.138621 26906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:02:52.138779 26906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "4828db66bbd44453a01e1e0c8c4d935b"
format_stamp: "Formatted at 2026-09-29 03:02:52 on dist-test-slave-m1rv"
server_key: "a9a03ea4cdceb033622097f27d146ece"
server_key_iv: "66c1ba2e7bbfd122116b078744dd09d9"
server_key_version: "encryptionkey@0"
I20260929 03:02:52.138978 26906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:52.170010 26906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:52.171087 26906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:52.171494 26906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:52.172844 26906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:52.175225 26906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:52.175345 26906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:52.175473 26906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:52.175565 26906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:52.214937 26906 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:37113
I20260929 03:02:52.215037 27032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:37113 every 8 connection(s)
I20260929 03:02:52.216343 26906 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:02:52.217506 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 26906
I20260929 03:02:52.217727 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 03:02:52.218269 9025 external_mini_cluster.cc:1428] Setting key 838a148ee7e49a19480abdd8573e44e4
I20260929 03:02:52.221305 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:40733
--builtin_ntp_servers=127.8.208.84:39693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 03:02:52.230494 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:52.219443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:44879 (local address 127.8.208.126:40733)
0929 03:02:52.219656 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:52.219663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:52.221425 (+ 1762us) server_negotiation.cc:407] Connection header received
0929 03:02:52.223320 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:52.223327 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:52.223435 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:52.223559 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:52.225057 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.225917 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.227563 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.227972 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.228127 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:52.229718 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:52.229725 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:52.229731 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:02:52.229755 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:52.229795 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:02:52.229810 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:52.229916 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:52.230187 (+ 271us) server_negotiation.cc:299] Negotiation successful
0929 03:02:52.230264 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 03:02:52.231689 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:52.220112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:40733 (local address 127.8.208.66:44879)
0929 03:02:52.221263 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:52.221315 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:52.223029 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:52.223804 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:52.223813 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:52.223929 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:52.224805 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:52.224838 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.226117 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.226121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:52.227401 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:52.227414 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.228211 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.228219 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:52.228473 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:52.229297 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:02:52.229303 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:52.229379 (+ 76us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:02:52.229489 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:52.229932 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:52.229945 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:52.229956 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 03:02:52.230182 (+ 226us) client_negotiation.cc:240] Negotiation successful
0929 03:02:52.230637 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":367,"threads_started":1}
I20260929 03:02:52.233289 27033 heartbeater.cc:347] Connected to a master server at 127.8.208.126:40733
I20260929 03:02:52.233592 27033 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:52.234360 27033 heartbeater.cc:511] Master 127.8.208.126:40733 requested a full tablet report, sending...
I20260929 03:02:52.235977 26716 ts_manager.cc:194] Registered new tserver with Master: 4828db66bbd44453a01e1e0c8c4d935b (127.8.208.66:37113)
I20260929 03:02:52.236919 26716 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.208.66:44879
W20260929 03:02:52.550349 27036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:52.550628 27036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:52.550810 27036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:52.558089 27036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:52.558180 27036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:52.558357 27036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:02:52.568811 27036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:39693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:40733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:52.570178 27036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:52.571929 27036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:52.584975 27041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:52.585003 27042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:52.584909 27044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:52.585785 27036 server_base.cc:1034] running on GCE node
I20260929 03:02:52.586366 27036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:52.587292 27036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:52.588486 27036 hybrid_clock.cc:648] HybridClock initialized: now 1790650972588425 us; error 53 us; skew 500 ppm
I20260929 03:02:52.588793 27036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 03:02:52.596745 27036 webserver.cc:460] Webserver started at http://127.8.208.67:40381/ using document root <none> and password file <none>
I20260929 03:02:52.597575 27036 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:52.597692 27036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:52.598174 27036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:52.601006 27036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "9705a9a897214f4cabee8be2c2ee8c6c"
format_stamp: "Formatted at 2026-09-29 03:02:52 on dist-test-slave-m1rv"
server_key: "ccf72b9b4eb06d8893670cacc2dd56d3"
server_key_iv: "30ddb32fcbb4bd31d594201a9f452c60"
server_key_version: "encryptionkey@0"
I20260929 03:02:52.601869 27036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "9705a9a897214f4cabee8be2c2ee8c6c"
format_stamp: "Formatted at 2026-09-29 03:02:52 on dist-test-slave-m1rv"
server_key: "ccf72b9b4eb06d8893670cacc2dd56d3"
server_key_iv: "30ddb32fcbb4bd31d594201a9f452c60"
server_key_version: "encryptionkey@0"
I20260929 03:02:52.607101 27036 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 03:02:52.610813 27050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:52.612264 27036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:02:52.612468 27036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "9705a9a897214f4cabee8be2c2ee8c6c"
format_stamp: "Formatted at 2026-09-29 03:02:52 on dist-test-slave-m1rv"
server_key: "ccf72b9b4eb06d8893670cacc2dd56d3"
server_key_iv: "30ddb32fcbb4bd31d594201a9f452c60"
server_key_version: "encryptionkey@0"
I20260929 03:02:52.612707 27036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:52.632459 27036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:52.633431 27036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:52.633774 27036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:52.634690 27036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:52.636094 27036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:52.636186 27036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:52.636266 27036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:52.636325 27036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:52.671224 27036 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:35509
I20260929 03:02:52.671327 27162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:35509 every 8 connection(s)
I20260929 03:02:52.672675 27036 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:02:52.679986 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 27036
I20260929 03:02:52.680183 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 03:02:52.680789 9025 external_mini_cluster.cc:1428] Setting key e6dd01b1649a47a2b94d2686e8f77cf9
I20260929 03:02:52.686403 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:52.675668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:44549 (local address 127.8.208.126:40733)
0929 03:02:52.675867 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:52.675871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:52.677321 (+ 1450us) server_negotiation.cc:407] Connection header received
0929 03:02:52.679132 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:52.679136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:52.679205 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:52.679294 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:52.680496 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.681491 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.683652 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.684154 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.684253 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:52.685732 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:52.685736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:52.685740 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:02:52.685757 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:52.685788 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:02:52.685799 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:52.685872 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:52.686077 (+ 205us) server_negotiation.cc:299] Negotiation successful
0929 03:02:52.686154 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 03:02:52.687443 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:52.676146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:40733 (local address 127.8.208.67:44549)
0929 03:02:52.677099 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:52.677141 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:52.678875 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:52.679500 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:52.679508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:52.679595 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:52.680313 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:52.680333 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.681648 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.681654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:52.683510 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:52.683528 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.684352 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.684357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:52.684603 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:52.685436 (+ 833us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:02:52.685439 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:52.685480 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:02:52.685564 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:52.685888 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:52.685896 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:52.685905 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 03:02:52.686053 (+ 148us) client_negotiation.cc:240] Negotiation successful
0929 03:02:52.686352 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":282,"threads_started":1}
I20260929 03:02:52.688746 27163 heartbeater.cc:347] Connected to a master server at 127.8.208.126:40733
I20260929 03:02:52.689060 27163 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:52.689884 27163 heartbeater.cc:511] Master 127.8.208.126:40733 requested a full tablet report, sending...
I20260929 03:02:52.691367 26716 ts_manager.cc:194] Registered new tserver with Master: 9705a9a897214f4cabee8be2c2ee8c6c (127.8.208.67:35509)
I20260929 03:02:52.692313 26716 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.208.67:44549
I20260929 03:02:52.696146 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:02:52.706108 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:52.700168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43352 (local address 127.8.208.126:40733)
0929 03:02:52.700377 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:52.700386 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:52.700742 (+ 356us) server_negotiation.cc:407] Connection header received
0929 03:02:52.700926 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:52.700931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:52.701007 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:52.701110 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:52.701605 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.702305 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.703694 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.704084 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.704231 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:52.705425 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:52.705430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:52.705433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:02:52.705449 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:52.705478 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:02:52.705489 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:52.705585 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:52.705726 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 03:02:52.705816 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 03:02:52.709734 26716 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:02:52.716754 26716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:02:52.743916 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:52.734010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:38385 (local address 127.0.0.1:47792)
0929 03:02:52.734504 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:52.734520 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:52.734631 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:52.735746 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:52.735752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:52.735778 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:52.736127 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:52.736141 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.737959 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.737965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:52.740733 (+ 2768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:52.740748 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.742780 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.742789 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:52.743078 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:52.743142 (+ 64us) client_negotiation.cc:769] Sending connection context
0929 03:02:52.743249 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 03:02:52.743444 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":70,"thread_start_us":1056,"threads_started":1}
I20260929 03:02:52.743944 27175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:52.732355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35509 (local address 127.0.0.1:53562)
0929 03:02:52.733039 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:52.733076 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:52.733270 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:52.734560 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:52.734569 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:52.734605 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:52.735079 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:52.735095 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.737673 (+ 2578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.737681 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:52.740735 (+ 3054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:52.740748 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.743113 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.743119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:52.743294 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:52.743320 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 03:02:52.743385 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 03:02:52.743485 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":176,"thread_start_us":688,"threads_started":1}
I20260929 03:02:52.744294 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:52.732899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53562 (local address 127.8.208.67:35509)
0929 03:02:52.733478 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:52.733489 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:52.733523 (+ 34us) server_negotiation.cc:407] Connection header received
0929 03:02:52.734025 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:52.734033 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:52.734310 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:52.735299 (+ 989us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:52.735587 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.737498 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.740988 (+ 3490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.742902 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.743062 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:52.743418 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:52.743499 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 03:02:52.743848 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":1917,"threads_started":1}
I20260929 03:02:52.745730 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:52.734606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.8.208.65:38385)
0929 03:02:52.735203 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:52.735212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:52.735245 (+ 33us) server_negotiation.cc:407] Connection header received
0929 03:02:52.735323 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:52.735330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:52.735532 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:52.736303 (+ 771us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:52.736544 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.737822 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.740996 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.742538 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.744702 (+ 2164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:52.745039 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:52.745109 (+ 70us) server_negotiation.cc:299] Negotiation successful
0929 03:02:52.745323 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":1417,"threads_started":1}
I20260929 03:02:52.753247 26902 heartbeater.cc:503] Master 127.8.208.126:40733 was elected leader, sending a full tablet report...
I20260929 03:02:52.755057 26837 tablet_service.cc:1467] Processing CreateTablet for tablet 1d38f6546d48431d821c3898915fa457 (DEFAULT_TABLE table=test-table [id=e397bc00df5c47dd8721dc04ddb5005e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:02:52.756524 27175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:52.735355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:37113 (local address 127.0.0.1:48678)
0929 03:02:52.744513 (+ 9158us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:52.744541 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:52.744655 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:52.750153 (+ 5498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:52.750158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:52.750187 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:52.750384 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:52.750397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.752254 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.752259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:52.754066 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:52.754081 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.755785 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.755793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:52.755982 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:52.756015 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 03:02:52.756105 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 03:02:52.756250 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9062,"thread_start_us":15148,"threads_started":1}
I20260929 03:02:52.756587 26837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d38f6546d48431d821c3898915fa457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:52.756263 27098 tablet_service.cc:1467] Processing CreateTablet for tablet 1d38f6546d48431d821c3898915fa457 (DEFAULT_TABLE table=test-table [id=e397bc00df5c47dd8721dc04ddb5005e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:02:52.757588 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:52.735861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48678 (local address 127.8.208.66:37113)
0929 03:02:52.749635 (+ 13774us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:52.749641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:52.749676 (+ 35us) server_negotiation.cc:407] Connection header received
0929 03:02:52.749763 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:52.749769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:52.749966 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:52.750175 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:52.750578 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.752084 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.754274 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:52.755603 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:52.756770 (+ 1167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:52.756987 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:52.757073 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 03:02:52.757252 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13624,"thread_start_us":13501,"threads_started":1}
I20260929 03:02:52.757793 27098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d38f6546d48431d821c3898915fa457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:52.765582 26968 tablet_service.cc:1467] Processing CreateTablet for tablet 1d38f6546d48431d821c3898915fa457 (DEFAULT_TABLE table=test-table [id=e397bc00df5c47dd8721dc04ddb5005e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:02:52.767040 26968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d38f6546d48431d821c3898915fa457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:52.774176 27182 tablet_bootstrap.cc:492] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c: Bootstrap starting.
I20260929 03:02:52.779146 27182 tablet_bootstrap.cc:654] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:52.780625 27182 log.cc:824] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:52.784745 27181 tablet_bootstrap.cc:492] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14: Bootstrap starting.
I20260929 03:02:52.785683 27184 tablet_bootstrap.cc:492] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b: Bootstrap starting.
I20260929 03:02:52.787019 27182 tablet_bootstrap.cc:492] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c: No bootstrap required, opened a new log
I20260929 03:02:52.787310 27182 ts_tablet_manager.cc:1397] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260929 03:02:52.788616 27181 tablet_bootstrap.cc:654] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:52.789485 27184 tablet_bootstrap.cc:654] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:52.790786 27181 log.cc:824] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:52.790762 27184 log.cc:824] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:52.798537 27182 raft_consensus.cc:348] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9705a9a897214f4cabee8be2c2ee8c6c" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35509 } } peers { permanent_uuid: "b1ec6127350d44d6af307b0fec0f7f14" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 38385 } } peers { permanent_uuid: "4828db66bbd44453a01e1e0c8c4d935b" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 37113 } }
I20260929 03:02:52.799139 27182 raft_consensus.cc:374] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:52.799324 27182 raft_consensus.cc:729] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9705a9a897214f4cabee8be2c2ee8c6c, State: Initialized, Role: FOLLOWER
I20260929 03:02:52.800201 27182 consensus_queue.cc:260] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9705a9a897214f4cabee8be2c2ee8c6c" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35509 } } peers { permanent_uuid: "b1ec6127350d44d6af307b0fec0f7f14" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 38385 } } peers { permanent_uuid: "4828db66bbd44453a01e1e0c8c4d935b" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 37113 } }
I20260929 03:02:52.801054 27184 tablet_bootstrap.cc:492] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b: No bootstrap required, opened a new log
I20260929 03:02:52.801056 27181 tablet_bootstrap.cc:492] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14: No bootstrap required, opened a new log
I20260929 03:02:52.801367 27181 ts_tablet_manager.cc:1397] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260929 03:02:52.801367 27184 ts_tablet_manager.cc:1397] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20260929 03:02:52.801560 27163 heartbeater.cc:503] Master 127.8.208.126:40733 was elected leader, sending a full tablet report...
I20260929 03:02:52.803995 27182 ts_tablet_manager.cc:1428] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260929 03:02:52.812142 27181 raft_consensus.cc:348] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9705a9a897214f4cabee8be2c2ee8c6c" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35509 } } peers { permanent_uuid: "b1ec6127350d44d6af307b0fec0f7f14" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 38385 } } peers { permanent_uuid: "4828db66bbd44453a01e1e0c8c4d935b" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 37113 } }
I20260929 03:02:52.812140 27184 raft_consensus.cc:348] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9705a9a897214f4cabee8be2c2ee8c6c" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35509 } } peers { permanent_uuid: "b1ec6127350d44d6af307b0fec0f7f14" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 38385 } } peers { permanent_uuid: "4828db66bbd44453a01e1e0c8c4d935b" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 37113 } }
I20260929 03:02:52.812615 27184 raft_consensus.cc:374] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:52.812615 27181 raft_consensus.cc:374] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:52.812718 27181 raft_consensus.cc:729] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1ec6127350d44d6af307b0fec0f7f14, State: Initialized, Role: FOLLOWER
I20260929 03:02:52.812762 27184 raft_consensus.cc:729] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4828db66bbd44453a01e1e0c8c4d935b, State: Initialized, Role: FOLLOWER
I20260929 03:02:52.813444 27181 consensus_queue.cc:260] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9705a9a897214f4cabee8be2c2ee8c6c" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35509 } } peers { permanent_uuid: "b1ec6127350d44d6af307b0fec0f7f14" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 38385 } } peers { permanent_uuid: "4828db66bbd44453a01e1e0c8c4d935b" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 37113 } }
I20260929 03:02:52.813469 27184 consensus_queue.cc:260] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9705a9a897214f4cabee8be2c2ee8c6c" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35509 } } peers { permanent_uuid: "b1ec6127350d44d6af307b0fec0f7f14" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 38385 } } peers { permanent_uuid: "4828db66bbd44453a01e1e0c8c4d935b" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 37113 } }
I20260929 03:02:52.814574 27033 heartbeater.cc:503] Master 127.8.208.126:40733 was elected leader, sending a full tablet report...
I20260929 03:02:52.815374 27184 ts_tablet_manager.cc:1428] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260929 03:02:52.815572 27181 ts_tablet_manager.cc:1428] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
W20260929 03:02:52.924723 27164 tablet.cc:2369] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:02:52.968894 27034 tablet.cc:2369] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:02:52.981487 26903 tablet.cc:2369] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:02:53.045341 27189 raft_consensus.cc:482] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:02:53.045693 27189 raft_consensus.cc:504] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9705a9a897214f4cabee8be2c2ee8c6c" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35509 } } peers { permanent_uuid: "b1ec6127350d44d6af307b0fec0f7f14" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 38385 } } peers { permanent_uuid: "4828db66bbd44453a01e1e0c8c4d935b" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 37113 } }
I20260929 03:02:53.047282 27189 leader_election.cc:288] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9705a9a897214f4cabee8be2c2ee8c6c (127.8.208.67:35509), b1ec6127350d44d6af307b0fec0f7f14 (127.8.208.65:38385)
I20260929 03:02:53.054395 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:53.047738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:48561 (local address 127.8.208.67:35509)
0929 03:02:53.047919 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:53.047923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:53.048296 (+ 373us) server_negotiation.cc:407] Connection header received
0929 03:02:53.048510 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:53.048517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:53.048600 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:53.048698 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:53.049484 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.050399 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.052475 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.053424 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.053559 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:53.053607 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:53.054004 (+ 397us) server_negotiation.cc:299] Negotiation successful
0929 03:02:53.054146 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 03:02:53.054421 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:53.047519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:35509 (local address 127.8.208.66:48561)
0929 03:02:53.048188 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:53.048208 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:53.048353 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:53.048850 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:53.048855 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:53.048877 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:53.049298 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:53.049311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.050530 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.050536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:53.052276 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:53.052288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.053624 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.053630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:53.053822 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:53.053855 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 03:02:53.053955 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 03:02:53.054044 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":102,"thread_start_us":340,"threads_started":1}
I20260929 03:02:53.054510 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:53.048548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:42955 (local address 127.8.208.65:38385)
0929 03:02:53.048752 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:53.048757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:53.049049 (+ 292us) server_negotiation.cc:407] Connection header received
0929 03:02:53.049250 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:53.049257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:53.049346 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:53.049439 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:53.050000 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.050834 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.052729 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.053632 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.053800 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:53.053862 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:53.054153 (+ 291us) server_negotiation.cc:299] Negotiation successful
0929 03:02:53.054273 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 03:02:53.054610 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:53.048299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:38385 (local address 127.8.208.66:42955)
0929 03:02:53.048931 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:53.048944 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:53.049090 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:53.049465 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:53.049469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:53.049492 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:53.049856 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:53.049867 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.050980 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.050984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:53.052536 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:53.052551 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.053811 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.053816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:53.053986 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:53.054017 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 03:02:53.054103 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 03:02:53.054211 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":65,"spinlock_wait_cycles":23424,"thread_start_us":361,"threads_started":1}
I20260929 03:02:53.055603 27118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d38f6546d48431d821c3898915fa457" candidate_uuid: "4828db66bbd44453a01e1e0c8c4d935b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9705a9a897214f4cabee8be2c2ee8c6c" is_pre_election: true
I20260929 03:02:53.055668 26857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d38f6546d48431d821c3898915fa457" candidate_uuid: "4828db66bbd44453a01e1e0c8c4d935b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1ec6127350d44d6af307b0fec0f7f14" is_pre_election: true
I20260929 03:02:53.056110 27118 raft_consensus.cc:2454] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4828db66bbd44453a01e1e0c8c4d935b in term 0.
I20260929 03:02:53.056231 26857 raft_consensus.cc:2454] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4828db66bbd44453a01e1e0c8c4d935b in term 0.
I20260929 03:02:53.056814 26923 leader_election.cc:302] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4828db66bbd44453a01e1e0c8c4d935b, 9705a9a897214f4cabee8be2c2ee8c6c; no voters:
I20260929 03:02:53.057215 27189 raft_consensus.cc:2781] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:02:53.057367 27189 raft_consensus.cc:482] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:02:53.057520 27189 raft_consensus.cc:3037] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:53.059705 27189 raft_consensus.cc:504] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9705a9a897214f4cabee8be2c2ee8c6c" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35509 } } peers { permanent_uuid: "b1ec6127350d44d6af307b0fec0f7f14" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 38385 } } peers { permanent_uuid: "4828db66bbd44453a01e1e0c8c4d935b" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 37113 } }
I20260929 03:02:53.060259 27189 leader_election.cc:288] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [CANDIDATE]: Term 1 election: Requested vote from peers 9705a9a897214f4cabee8be2c2ee8c6c (127.8.208.67:35509), b1ec6127350d44d6af307b0fec0f7f14 (127.8.208.65:38385)
I20260929 03:02:53.060684 27118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d38f6546d48431d821c3898915fa457" candidate_uuid: "4828db66bbd44453a01e1e0c8c4d935b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9705a9a897214f4cabee8be2c2ee8c6c"
I20260929 03:02:53.060755 26857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d38f6546d48431d821c3898915fa457" candidate_uuid: "4828db66bbd44453a01e1e0c8c4d935b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1ec6127350d44d6af307b0fec0f7f14"
I20260929 03:02:53.060873 27118 raft_consensus.cc:3037] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:53.060977 26857 raft_consensus.cc:3037] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:53.063114 27118 raft_consensus.cc:2454] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4828db66bbd44453a01e1e0c8c4d935b in term 1.
I20260929 03:02:53.063169 26857 raft_consensus.cc:2454] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4828db66bbd44453a01e1e0c8c4d935b in term 1.
I20260929 03:02:53.063510 26923 leader_election.cc:302] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4828db66bbd44453a01e1e0c8c4d935b, 9705a9a897214f4cabee8be2c2ee8c6c; no voters:
I20260929 03:02:53.063803 27189 raft_consensus.cc:2781] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:02:53.064254 27189 raft_consensus.cc:686] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 1 LEADER]: Becoming Leader. State: Replica: 4828db66bbd44453a01e1e0c8c4d935b, State: Running, Role: LEADER
I20260929 03:02:53.064734 27189 consensus_queue.cc:237] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9705a9a897214f4cabee8be2c2ee8c6c" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35509 } } peers { permanent_uuid: "b1ec6127350d44d6af307b0fec0f7f14" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 38385 } } peers { permanent_uuid: "4828db66bbd44453a01e1e0c8c4d935b" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 37113 } }
I20260929 03:02:53.069576 26716 catalog_manager.cc:5495] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4828db66bbd44453a01e1e0c8c4d935b (127.8.208.66). New cstate: current_term: 1 leader_uuid: "4828db66bbd44453a01e1e0c8c4d935b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9705a9a897214f4cabee8be2c2ee8c6c" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1ec6127350d44d6af307b0fec0f7f14" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 38385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4828db66bbd44453a01e1e0c8c4d935b" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 37113 } health_report { overall_health: HEALTHY } } }
I20260929 03:02:53.126643 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:53.121407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48680 (local address 127.8.208.66:37113)
0929 03:02:53.121609 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:53.121615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:53.121637 (+ 22us) server_negotiation.cc:407] Connection header received
0929 03:02:53.121788 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:53.121792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:53.121884 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:53.122047 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:02:53.122404 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.123090 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.124492 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.124894 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.125036 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:53.125582 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:02:53.126080 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:02:53.126236 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:53.126295 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 03:02:53.126370 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 03:02:53.134595 27118 raft_consensus.cc:1261] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c [term 1 FOLLOWER]: Refusing update from remote peer 4828db66bbd44453a01e1e0c8c4d935b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:02:53.134595 26857 raft_consensus.cc:1261] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14 [term 1 FOLLOWER]: Refusing update from remote peer 4828db66bbd44453a01e1e0c8c4d935b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:02:53.135581 27192 consensus_queue.cc:1035] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9705a9a897214f4cabee8be2c2ee8c6c" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 35509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:02:53.135932 27189 consensus_queue.cc:1035] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1ec6127350d44d6af307b0fec0f7f14" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 38385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:02:53.147351 27195 mvcc.cc:205] Tried to move back new op lower bound from 7334506385948585984 to 7334506385676840960. Current Snapshot: MvccSnapshot[applied={T|T < 7334506385948585984}]
I20260929 03:02:53.163841 26715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:02:53.164186 26715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:02:53.168689 26715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 276a0cbc1b3e4b849a81324b81931434: Sending DeleteTablet for 3 replicas of tablet 1d38f6546d48431d821c3898915fa457
I20260929 03:02:53.170015 26968 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d38f6546d48431d821c3898915fa457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:02:53 UTC) from {username='slave'} at 127.0.0.1:48678
I20260929 03:02:53.170017 26837 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d38f6546d48431d821c3898915fa457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:02:53 UTC) from {username='slave'} at 127.0.0.1:47792
I20260929 03:02:53.171072 27098 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d38f6546d48431d821c3898915fa457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:02:53 UTC) from {username='slave'} at 127.0.0.1:53562
I20260929 03:02:53.174106 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 26775
I20260929 03:02:53.181422 27207 tablet_replica.cc:330] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c: stopping tablet replica
I20260929 03:02:53.181998 27207 raft_consensus.cc:2229] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:02:53.182658 27207 raft_consensus.cc:2258] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:02:53.183496 27205 tablet_replica.cc:330] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b: stopping tablet replica
I20260929 03:02:53.183717 27206 tablet_replica.cc:330] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14: stopping tablet replica
I20260929 03:02:53.183895 27205 raft_consensus.cc:2229] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:02:53.184101 27206 raft_consensus.cc:2229] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:02:53.184600 27206 raft_consensus.cc:2258] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:02:53.184645 27205 raft_consensus.cc:2258] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:02:53.186868 27205 ts_tablet_manager.cc:1905] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:02:53.186878 27206 ts_tablet_manager.cc:1905] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:02:53.186913 27207 ts_tablet_manager.cc:1905] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:02:53.193842 27207 ts_tablet_manager.cc:1918] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:02:53.193842 27206 ts_tablet_manager.cc:1918] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:02:53.194027 27206 log.cc:1196] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/1d38f6546d48431d821c3898915fa457
I20260929 03:02:53.194058 27207 log.cc:1196] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/1d38f6546d48431d821c3898915fa457
I20260929 03:02:53.194660 27206 ts_tablet_manager.cc:1939] T 1d38f6546d48431d821c3898915fa457 P b1ec6127350d44d6af307b0fec0f7f14: Deleting consensus metadata
I20260929 03:02:53.194660 27207 ts_tablet_manager.cc:1939] T 1d38f6546d48431d821c3898915fa457 P 9705a9a897214f4cabee8be2c2ee8c6c: Deleting consensus metadata
I20260929 03:02:53.196566 26702 catalog_manager.cc:4841] TS 9705a9a897214f4cabee8be2c2ee8c6c (127.8.208.67:35509): tablet 1d38f6546d48431d821c3898915fa457 (table test-table [id=e397bc00df5c47dd8721dc04ddb5005e]) successfully deleted
I20260929 03:02:53.196817 27205 ts_tablet_manager.cc:1918] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:02:53.197000 27205 log.cc:1196] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/1d38f6546d48431d821c3898915fa457
I20260929 03:02:53.197218 26702 catalog_manager.cc:4841] TS b1ec6127350d44d6af307b0fec0f7f14 (127.8.208.65:38385): tablet 1d38f6546d48431d821c3898915fa457 (table test-table [id=e397bc00df5c47dd8721dc04ddb5005e]) successfully deleted
I20260929 03:02:53.197597 27205 ts_tablet_manager.cc:1939] T 1d38f6546d48431d821c3898915fa457 P 4828db66bbd44453a01e1e0c8c4d935b: Deleting consensus metadata
I20260929 03:02:53.199316 26702 catalog_manager.cc:4841] TS 4828db66bbd44453a01e1e0c8c4d935b (127.8.208.66:37113): tablet 1d38f6546d48431d821c3898915fa457 (table test-table [id=e397bc00df5c47dd8721dc04ddb5005e]) successfully deleted
I20260929 03:02:53.199782 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:53.175563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.8.208.65:38385)
0929 03:02:53.175763 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:53.175770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:53.180691 (+ 4921us) server_negotiation.cc:407] Connection header received
0929 03:02:53.180909 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:53.180915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:53.181025 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:53.181131 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:53.182576 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.183785 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.186114 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.186764 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.186873 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:53.197908 (+ 11035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:53.197917 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:53.197931 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:02:53.198611 (+ 680us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:53.198668 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:02:53.198688 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:53.198848 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:53.199340 (+ 492us) server_negotiation.cc:299] Negotiation successful
0929 03:02:53.199513 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 03:02:53.200459 26897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:53.308446 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 26775
I20260929 03:02:53.326532 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 26906
I20260929 03:02:53.334088 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:53.327450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48684 (local address 127.8.208.66:37113)
0929 03:02:53.327654 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:53.327661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:53.327681 (+ 20us) server_negotiation.cc:407] Connection header received
0929 03:02:53.327886 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:53.327892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:53.327992 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:53.328118 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:53.328513 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.329395 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.330702 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.331127 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.331263 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:53.332541 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:53.332550 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:53.332564 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:02:53.333295 (+ 731us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:53.333348 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:02:53.333375 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:53.333493 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:53.333648 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 03:02:53.333747 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 03:02:53.334878 27028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:53.452088 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 26906
I20260929 03:02:53.470414 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 27036
I20260929 03:02:53.481395 27176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:53.475012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53572 (local address 127.8.208.67:35509)
0929 03:02:53.475190 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:53.475197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:53.475250 (+ 53us) server_negotiation.cc:407] Connection header received
0929 03:02:53.475501 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:53.475505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:53.475585 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:53.475693 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:53.476118 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.476867 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.478292 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:53.478687 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:53.478823 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:53.479986 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:53.479995 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:53.480009 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:02:53.480557 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:53.480605 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:02:53.480624 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:53.480746 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:53.480878 (+ 132us) server_negotiation.cc:299] Negotiation successful
0929 03:02:53.481046 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:02:53.482040 27158 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:53.571861 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 27036
I20260929 03:02:53.588390 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 26685
I20260929 03:02:53.589473 26746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:53.699373 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 26685
2026-09-29T03:02:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3045 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:02:53 dist-test-slave-m1rv krb5kdc[27215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:02:53 dist-test-slave-m1rv krb5kdc[27215](info): set up 2 sockets
Sep 29 03:02:53 dist-test-slave-m1rv krb5kdc[27215](info): commencing operation
krb5kdc: starting...
W20260929 03:02:54.817418 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:02:54 dist-test-slave-m1rv krb5kdc[27215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:02:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:02:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:02:55.060425 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:46727
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:46727
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:02:55.386636 27228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:55.386978 27228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:55.387152 27228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:55.394263 27228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:02:55.394416 27228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:55.394531 27228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:55.394636 27228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:02:55.394704 27228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:02:55.404423 27228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:46727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:46727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:55.405830 27228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:55.407228 27228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:55.420074 27234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:55.420147 27233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:55.420147 27236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:55.421651 27228 server_base.cc:1034] running on GCE node
I20260929 03:02:55.422495 27228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:55.423962 27228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:55.425251 27228 hybrid_clock.cc:648] HybridClock initialized: now 1790650975425204 us; error 54 us; skew 500 ppm
I20260929 03:02:55.425638 27228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:55 dist-test-slave-m1rv krb5kdc[27215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650975, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:55.428720 27228 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:02:55.430279 27228 webserver.cc:460] Webserver started at http://127.8.208.126:45817/ using document root <none> and password file <none>
I20260929 03:02:55.430980 27228 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:55.431137 27228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:55.431452 27228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:55.434528 27228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "92d2034650fc44d195f3081f2a2bff95"
format_stamp: "Formatted at 2026-09-29 03:02:55 on dist-test-slave-m1rv"
server_key: "08642de2252e04892a5c2a9527f1bee8"
server_key_iv: "5375685b9a117b039c0fd4f457b157cf"
server_key_version: "encryptionkey@0"
I20260929 03:02:55.435310 27228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "92d2034650fc44d195f3081f2a2bff95"
format_stamp: "Formatted at 2026-09-29 03:02:55 on dist-test-slave-m1rv"
server_key: "08642de2252e04892a5c2a9527f1bee8"
server_key_iv: "5375685b9a117b039c0fd4f457b157cf"
server_key_version: "encryptionkey@0"
I20260929 03:02:55.441090 27228 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260929 03:02:55.444625 27243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:55.446079 27228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 03:02:55.446233 27228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "92d2034650fc44d195f3081f2a2bff95"
format_stamp: "Formatted at 2026-09-29 03:02:55 on dist-test-slave-m1rv"
server_key: "08642de2252e04892a5c2a9527f1bee8"
server_key_iv: "5375685b9a117b039c0fd4f457b157cf"
server_key_version: "encryptionkey@0"
I20260929 03:02:55.446398 27228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:55.465822 27228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:55.469738 27228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:55.469976 27228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:55.488395 27228 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:46727
I20260929 03:02:55.488420 27294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:46727 every 8 connection(s)
I20260929 03:02:55.489777 27228 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:02:55.493410 27295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:55.493827 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 27228
I20260929 03:02:55.494028 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:02:55.494496 9025 external_mini_cluster.cc:1428] Setting key 224e07c80f042ea3007600bf0ddb94c2
I20260929 03:02:55.507833 27295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95: Bootstrap starting.
Sep 29 03:02:55 dist-test-slave-m1rv krb5kdc[27215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:02:55.511121 27295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:55.512331 27295 log.cc:824] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:55.515795 27295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95: No bootstrap required, opened a new log
I20260929 03:02:55.520315 27298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:55.496447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54258 (local address 127.8.208.126:46727)
0929 03:02:55.497294 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:55.497309 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:55.497359 (+ 50us) server_negotiation.cc:407] Connection header received
0929 03:02:55.500058 (+ 2699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:55.500088 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:55.500643 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:55.501124 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:55.502531 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:55.504550 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:55.505993 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:55.507031 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:55.507208 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:55.511777 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:55.511787 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:55.511804 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:55.511851 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:55.515782 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:55.516545 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:55.516553 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:55.516561 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:55.516767 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:55.517162 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:55.517167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:55.517195 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:55.517871 (+ 676us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:55.518227 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:55.518515 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 03:02:55.518947 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":347,"threads_started":1}
I20260929 03:02:55.525224 27295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d2034650fc44d195f3081f2a2bff95" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46727 } }
I20260929 03:02:55.525565 27295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:55.525640 27295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d2034650fc44d195f3081f2a2bff95, State: Initialized, Role: FOLLOWER
I20260929 03:02:55.526150 27295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d2034650fc44d195f3081f2a2bff95" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46727 } }
I20260929 03:02:55.526350 27295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:02:55.526444 27295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:02:55.526562 27295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:55.528581 27295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d2034650fc44d195f3081f2a2bff95" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46727 } }
I20260929 03:02:55.528996 27295 leader_election.cc:302] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92d2034650fc44d195f3081f2a2bff95; no voters:
I20260929 03:02:55.529598 27295 leader_election.cc:288] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:02:55.529726 27300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:02:55.530766 27300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [term 1 LEADER]: Becoming Leader. State: Replica: 92d2034650fc44d195f3081f2a2bff95, State: Running, Role: LEADER
I20260929 03:02:55.531322 27300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d2034650fc44d195f3081f2a2bff95" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46727 } }
I20260929 03:02:55.531652 27295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:02:55.534010 27301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92d2034650fc44d195f3081f2a2bff95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d2034650fc44d195f3081f2a2bff95" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46727 } } }
I20260929 03:02:55.534410 27301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [sys.catalog]: This master's current role is: LEADER
I20260929 03:02:55.534498 27302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92d2034650fc44d195f3081f2a2bff95. Latest consensus state: current_term: 1 leader_uuid: "92d2034650fc44d195f3081f2a2bff95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92d2034650fc44d195f3081f2a2bff95" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 46727 } } }
I20260929 03:02:55.534865 27302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95 [sys.catalog]: This master's current role is: LEADER
I20260929 03:02:55.537765 27309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:02:55.542094 27309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:02:55.550261 27309 catalog_manager.cc:1347] Generated new cluster ID: dd651f5d878349819473a061640779a3
I20260929 03:02:55.550392 27309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:02:55.593897 27309 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:02:55.595263 27309 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:02:55.602542 27309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95: Generated new TSK 0
I20260929 03:02:55.603379 27309 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 03:02:55.676551 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46727
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:02:56.011049 27321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:56.011333 27321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:56.011515 27321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:56.019208 27321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:56.019320 27321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:56.019520 27321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:02:56.029589 27321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:56.030901 27321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:56.032338 27321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:56.045940 27329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:56.045969 27327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:56.045969 27326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:56.046602 27321 server_base.cc:1034] running on GCE node
I20260929 03:02:56.047261 27321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:56.048179 27321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:56.049364 27321 hybrid_clock.cc:648] HybridClock initialized: now 1790650976049316 us; error 48 us; skew 500 ppm
I20260929 03:02:56.049666 27321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:56 dist-test-slave-m1rv krb5kdc[27215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650976, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:56.052984 27321 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 03:02:56.054488 27321 webserver.cc:460] Webserver started at http://127.8.208.65:37725/ using document root <none> and password file <none>
I20260929 03:02:56.055495 27321 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:56.055636 27321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:56.055945 27321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:56.058881 27321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "440f7b035d0a4a9ea124d432de417273"
format_stamp: "Formatted at 2026-09-29 03:02:56 on dist-test-slave-m1rv"
server_key: "2e7942d7f675f4e4c6a948fd9281845b"
server_key_iv: "1d876215aa124dbd1b64e9121454a316"
server_key_version: "encryptionkey@0"
I20260929 03:02:56.059652 27321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "440f7b035d0a4a9ea124d432de417273"
format_stamp: "Formatted at 2026-09-29 03:02:56 on dist-test-slave-m1rv"
server_key: "2e7942d7f675f4e4c6a948fd9281845b"
server_key_iv: "1d876215aa124dbd1b64e9121454a316"
server_key_version: "encryptionkey@0"
I20260929 03:02:56.064687 27321 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 03:02:56.068037 27336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:56.069350 27321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 03:02:56.069511 27321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "440f7b035d0a4a9ea124d432de417273"
format_stamp: "Formatted at 2026-09-29 03:02:56 on dist-test-slave-m1rv"
server_key: "2e7942d7f675f4e4c6a948fd9281845b"
server_key_iv: "1d876215aa124dbd1b64e9121454a316"
server_key_version: "encryptionkey@0"
I20260929 03:02:56.069728 27321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:56.091423 27321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:56.095675 27321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:56.095988 27321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:56.097021 27321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:56.098429 27321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:56.098547 27321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:56.098696 27321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:56.098776 27321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:56.132946 27321 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:35593
I20260929 03:02:56.133039 27448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:35593 every 8 connection(s)
I20260929 03:02:56.134298 27321 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:02:56.143344 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 27321
I20260929 03:02:56.143551 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 03:02:56.144110 9025 external_mini_cluster.cc:1428] Setting key 045368fddc5fdeceec8362d7b8abae71
Sep 29 03:02:56 dist-test-slave-m1rv krb5kdc[27215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650976, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:02:56.156311 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:56.137086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:38277 (local address 127.8.208.126:46727)
0929 03:02:56.137592 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:56.137598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:56.138497 (+ 899us) server_negotiation.cc:407] Connection header received
0929 03:02:56.140292 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:56.140296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:56.140365 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:56.140461 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:56.142099 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:56.142819 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:56.145118 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:56.145618 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:56.145772 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:56.150184 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:56.150189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:56.150192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:56.150226 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:56.153013 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:56.153778 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:56.153783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:56.153785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:56.153903 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:56.154340 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:56.154346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:56.154353 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:56.154746 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:56.154908 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:56.155679 (+ 771us) server_negotiation.cc:299] Negotiation successful
0929 03:02:56.155939 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":279,"threads_started":1}
I20260929 03:02:56.157447 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:56.137568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46727 (local address 127.8.208.65:38277)
0929 03:02:56.138323 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:56.138368 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:56.140035 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:56.140661 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:56.140669 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:56.141218 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:56.141933 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:56.141962 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:56.143036 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:56.143042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:56.144942 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:56.144960 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:56.145866 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:56.145874 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:56.146179 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:56.146970 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:02:56.147013 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:56.149906 (+ 2893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:56.153159 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:56.153189 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:56.153210 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:56.153640 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:56.154013 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:56.154027 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:56.154031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:56.154215 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:56.154898 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:56.154906 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:56.155245 (+ 339us) client_negotiation.cc:769] Sending connection context
0929 03:02:56.155562 (+ 317us) client_negotiation.cc:240] Negotiation successful
0929 03:02:56.156107 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":303,"threads_started":1}
I20260929 03:02:56.159122 27449 heartbeater.cc:347] Connected to a master server at 127.8.208.126:46727
I20260929 03:02:56.159436 27449 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:56.160284 27449 heartbeater.cc:511] Master 127.8.208.126:46727 requested a full tablet report, sending...
I20260929 03:02:56.162869 27260 ts_manager.cc:194] Registered new tserver with Master: 440f7b035d0a4a9ea124d432de417273 (127.8.208.65:35593)
I20260929 03:02:56.164318 27260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:38277
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 03:02:56.201160 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46727
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:02:56.535420 27455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:56.535727 27455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:56.535917 27455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:56.543143 27455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:56.543309 27455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:56.543489 27455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:02:56.553048 27455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:56.554316 27455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:56.555521 27455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:56.568702 27460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:56.568774 27461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:56.568748 27463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:56.569644 27455 server_base.cc:1034] running on GCE node
I20260929 03:02:56.570255 27455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:56.571216 27455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:56.572450 27455 hybrid_clock.cc:648] HybridClock initialized: now 1790650976572405 us; error 62 us; skew 500 ppm
I20260929 03:02:56.572767 27455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:56 dist-test-slave-m1rv krb5kdc[27215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650976, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:56.576200 27455 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 03:02:56.577746 27455 webserver.cc:460] Webserver started at http://127.8.208.66:45379/ using document root <none> and password file <none>
I20260929 03:02:56.578636 27455 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:56.578794 27455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:56.579162 27455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:56.582953 27455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "5f6bd69b7f274dee845d943b6654fd64"
format_stamp: "Formatted at 2026-09-29 03:02:56 on dist-test-slave-m1rv"
server_key: "84ccaa915f600a7eb1ab5af69d150bea"
server_key_iv: "28c5b1cd2f1b3160d7675c51b5c1c334"
server_key_version: "encryptionkey@0"
I20260929 03:02:56.583962 27455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "5f6bd69b7f274dee845d943b6654fd64"
format_stamp: "Formatted at 2026-09-29 03:02:56 on dist-test-slave-m1rv"
server_key: "84ccaa915f600a7eb1ab5af69d150bea"
server_key_iv: "28c5b1cd2f1b3160d7675c51b5c1c334"
server_key_version: "encryptionkey@0"
I20260929 03:02:56.589936 27455 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260929 03:02:56.593145 27470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:56.594439 27455 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:02:56.594615 27455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "5f6bd69b7f274dee845d943b6654fd64"
format_stamp: "Formatted at 2026-09-29 03:02:56 on dist-test-slave-m1rv"
server_key: "84ccaa915f600a7eb1ab5af69d150bea"
server_key_iv: "28c5b1cd2f1b3160d7675c51b5c1c334"
server_key_version: "encryptionkey@0"
I20260929 03:02:56.594803 27455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:56.627511 27455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:56.632202 27455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:56.632490 27455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:56.633562 27455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:56.634948 27455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:56.635051 27455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:56.635217 27455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:56.635289 27455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:56.667676 27455 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:46021
I20260929 03:02:56.667724 27582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:46021 every 8 connection(s)
I20260929 03:02:56.668973 27455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:02:56.675640 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 27455
I20260929 03:02:56.675841 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 03:02:56.676313 9025 external_mini_cluster.cc:1428] Setting key aee680bb754a20549b8170dcb73f21c0
Sep 29 03:02:56 dist-test-slave-m1rv krb5kdc[27215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650976, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:02:56.690425 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:56.671614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:42561 (local address 127.8.208.126:46727)
0929 03:02:56.672089 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:56.672093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:56.673125 (+ 1032us) server_negotiation.cc:407] Connection header received
0929 03:02:56.675302 (+ 2177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:56.675308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:56.675406 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:56.675552 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:56.677716 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:56.678572 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:56.680129 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:56.680569 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:56.680696 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:56.684424 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:56.684434 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:56.684438 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:56.684476 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:56.687224 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:56.688146 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:56.688149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:56.688151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:56.688270 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:56.688736 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:56.688739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:56.688741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:56.688989 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:56.689104 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:56.689984 (+ 880us) server_negotiation.cc:299] Negotiation successful
0929 03:02:56.690158 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":216,"threads_started":1}
I20260929 03:02:56.691903 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:56.672171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46727 (local address 127.8.208.66:42561)
0929 03:02:56.672942 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:56.672986 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:56.675007 (+ 2021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:56.675849 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:56.675862 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:56.676762 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:56.677548 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:56.677566 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:56.678722 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:56.678729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:56.679987 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:56.679999 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:56.680800 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:56.680807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:56.681041 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:56.681808 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:02:56.681833 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:56.684196 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:56.687362 (+ 3166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:56.687374 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:56.687393 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:56.688026 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:56.688379 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:56.688387 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:56.688393 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:56.688626 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:56.689128 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:56.689138 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:56.689602 (+ 464us) client_negotiation.cc:769] Sending connection context
0929 03:02:56.689891 (+ 289us) client_negotiation.cc:240] Negotiation successful
0929 03:02:56.690452 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":297,"threads_started":1}
I20260929 03:02:56.693444 27583 heartbeater.cc:347] Connected to a master server at 127.8.208.126:46727
I20260929 03:02:56.693753 27583 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:56.694470 27583 heartbeater.cc:511] Master 127.8.208.126:46727 requested a full tablet report, sending...
I20260929 03:02:56.695941 27260 ts_manager.cc:194] Registered new tserver with Master: 5f6bd69b7f274dee845d943b6654fd64 (127.8.208.66:46021)
I20260929 03:02:56.697273 27260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:42561
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 03:02:56.730794 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46727
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:02:57.062568 27589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:57.062851 27589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:57.062983 27589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:57.070215 27589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:57.070335 27589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:57.070456 27589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:02:57.084281 27589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:46727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:57.085631 27589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:57.086977 27589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:57.100930 27594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:57.100989 27597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:57.101015 27595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:57.101742 27589 server_base.cc:1034] running on GCE node
I20260929 03:02:57.102385 27589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:57.103375 27589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:57.104602 27589 hybrid_clock.cc:648] HybridClock initialized: now 1790650977104555 us; error 39 us; skew 500 ppm
I20260929 03:02:57.104928 27589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:02:57 dist-test-slave-m1rv krb5kdc[27215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650977, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:02:57.108072 27589 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 03:02:57.109494 27589 webserver.cc:460] Webserver started at http://127.8.208.67:46139/ using document root <none> and password file <none>
I20260929 03:02:57.110121 27589 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:57.110216 27589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:57.110464 27589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:57.113340 27589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "f29670104c9f4cf6bf7f1aef54783503"
format_stamp: "Formatted at 2026-09-29 03:02:57 on dist-test-slave-m1rv"
server_key: "cb9e4b9663ec6deb18663c68cc33dfda"
server_key_iv: "ccbb9357312de119d1d7210f70fee371"
server_key_version: "encryptionkey@0"
I20260929 03:02:57.114068 27589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "f29670104c9f4cf6bf7f1aef54783503"
format_stamp: "Formatted at 2026-09-29 03:02:57 on dist-test-slave-m1rv"
server_key: "cb9e4b9663ec6deb18663c68cc33dfda"
server_key_iv: "ccbb9357312de119d1d7210f70fee371"
server_key_version: "encryptionkey@0"
I20260929 03:02:57.118868 27589 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:02:57.122148 27604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:57.123421 27589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 03:02:57.123581 27589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "f29670104c9f4cf6bf7f1aef54783503"
format_stamp: "Formatted at 2026-09-29 03:02:57 on dist-test-slave-m1rv"
server_key: "cb9e4b9663ec6deb18663c68cc33dfda"
server_key_iv: "ccbb9357312de119d1d7210f70fee371"
server_key_version: "encryptionkey@0"
I20260929 03:02:57.123780 27589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:57.141757 27589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:57.145901 27589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:57.146180 27589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:57.147162 27589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:57.148814 27589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:57.149053 27589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:57.149245 27589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:57.149327 27589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260929 03:02:57.167768 27449 heartbeater.cc:503] Master 127.8.208.126:46727 was elected leader, sending a full tablet report...
I20260929 03:02:57.184296 27589 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:42343
I20260929 03:02:57.184332 27716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:42343 every 8 connection(s)
I20260929 03:02:57.185705 27589 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:02:57.193624 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 27589
I20260929 03:02:57.193820 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 03:02:57.194276 9025 external_mini_cluster.cc:1428] Setting key e1b461bc49c647c1324c1642e619f5f0
Sep 29 03:02:57 dist-test-slave-m1rv krb5kdc[27215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650977, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:02:57.206193 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.188484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:59639 (local address 127.8.208.126:46727)
0929 03:02:57.188708 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:57.188713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:57.189942 (+ 1229us) server_negotiation.cc:407] Connection header received
0929 03:02:57.191792 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:57.191798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:57.191895 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:57.192009 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:57.193950 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.194734 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.196525 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.197033 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.197144 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.200587 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:57.200592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:57.200595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:57.200627 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:57.203389 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:57.204092 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:57.204096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:57.204098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:57.204218 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:57.204598 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:57.204601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:57.204603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:57.204871 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:57.204989 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:57.205774 (+ 785us) server_negotiation.cc:299] Negotiation successful
0929 03:02:57.205950 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260929 03:02:57.207181 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.188983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:46727 (local address 127.8.208.67:59639)
0929 03:02:57.189750 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.189797 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:57.191521 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:57.192247 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:57.192256 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:57.192914 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:57.193698 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:57.193727 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.194882 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.194888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:57.196338 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:57.196371 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.197290 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.197296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:57.197495 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.198159 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:02:57.198185 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:57.200393 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:57.203535 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:57.203541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:57.203554 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:57.203964 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:57.204310 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:02:57.204313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:02:57.204315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:02:57.204485 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:02:57.204974 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:57.204979 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:57.205426 (+ 447us) client_negotiation.cc:769] Sending connection context
0929 03:02:57.205664 (+ 238us) client_negotiation.cc:240] Negotiation successful
0929 03:02:57.206049 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":342,"threads_started":1}
I20260929 03:02:57.208758 27717 heartbeater.cc:347] Connected to a master server at 127.8.208.126:46727
I20260929 03:02:57.209136 27717 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:57.209947 27717 heartbeater.cc:511] Master 127.8.208.126:46727 requested a full tablet report, sending...
I20260929 03:02:57.211362 27260 ts_manager.cc:194] Registered new tserver with Master: f29670104c9f4cf6bf7f1aef54783503 (127.8.208.67:42343)
I20260929 03:02:57.212185 27260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:59639
I20260929 03:02:57.222220 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:02:57.240162 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.226436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.8.208.126:46727)
0929 03:02:57.226665 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:57.226670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:57.227167 (+ 497us) server_negotiation.cc:407] Connection header received
0929 03:02:57.227261 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:57.227265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:57.227338 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:57.227424 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:57.229002 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.229876 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.231164 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.231687 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.231792 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.234751 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:57.234756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:57.234759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:57.234790 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:57.237479 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:57.238282 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:57.238286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:57.238288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:57.238398 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:57.238790 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:57.238793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:57.238795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:57.239071 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:57.239274 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:57.239645 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 03:02:57.239767 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 03:02:57.244263 27260 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:02:57.251782 27260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:02:57.279235 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.264255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44192 (local address 127.8.208.67:42343)
0929 03:02:57.265765 (+ 1510us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:57.265775 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:57.265807 (+ 32us) server_negotiation.cc:407] Connection header received
0929 03:02:57.265905 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:57.265912 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:57.266132 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:57.266329 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:57.267181 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.270936 (+ 3755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.275807 (+ 4871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.277375 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.277514 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.277740 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:57.278645 (+ 905us) server_negotiation.cc:299] Negotiation successful
0929 03:02:57.278842 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1325,"thread_start_us":1175,"threads_started":1}
I20260929 03:02:57.279470 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.263894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:42343 (local address 127.0.0.1:44192)
0929 03:02:57.264911 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.264947 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:57.265111 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:57.266350 (+ 1239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:57.266358 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:57.266395 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:57.266957 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:57.266975 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.271134 (+ 4159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.271143 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:57.275633 (+ 4490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:57.275651 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.277888 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.277893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:57.278245 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.278426 (+ 181us) client_negotiation.cc:769] Sending connection context
0929 03:02:57.278576 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 03:02:57.278834 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"mutex_wait_us":2379,"thread_start_us":670,"threads_started":1}
I20260929 03:02:57.279568 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.264175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:46021 (local address 127.0.0.1:56310)
0929 03:02:57.266613 (+ 2438us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.266633 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:57.266777 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:57.268207 (+ 1430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:57.268214 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:57.268251 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:57.268904 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:57.268920 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.271529 (+ 2609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.271547 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:57.275223 (+ 3676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:57.275248 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.277876 (+ 2628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.277885 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:57.278242 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.278426 (+ 184us) client_negotiation.cc:769] Sending connection context
0929 03:02:57.278576 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 03:02:57.278834 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2297,"mutex_wait_us":94,"thread_start_us":2133,"threads_started":1}
I20260929 03:02:57.280236 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.264724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56310 (local address 127.8.208.66:46021)
0929 03:02:57.266108 (+ 1384us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:57.266120 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:57.266953 (+ 833us) server_negotiation.cc:407] Connection header received
0929 03:02:57.267091 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:57.267103 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:57.267510 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:57.267774 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:57.269162 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.271274 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.275536 (+ 4262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.277290 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.277475 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.277692 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:57.279738 (+ 2046us) server_negotiation.cc:299] Negotiation successful
0929 03:02:57.279926 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1205,"thread_start_us":1082,"threads_started":1}
I20260929 03:02:57.284085 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.266805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35593 (local address 127.0.0.1:51994)
0929 03:02:57.275839 (+ 9034us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.275854 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:57.275994 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:57.276580 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:57.276584 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:57.276626 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:57.276977 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:57.276992 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.278723 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.278728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:57.281019 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:57.281034 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.283290 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.283295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:57.283422 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.283457 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 03:02:57.283574 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 03:02:57.283719 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8901,"thread_start_us":8715,"threads_started":1}
I20260929 03:02:57.284365 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.267182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51994 (local address 127.8.208.65:35593)
0929 03:02:57.274294 (+ 7112us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:57.274302 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:57.275956 (+ 1654us) server_negotiation.cc:407] Connection header received
0929 03:02:57.276177 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:57.276183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:57.276382 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:57.276600 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:57.277287 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.278513 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.281228 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.282771 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.282916 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.283171 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:57.283630 (+ 459us) server_negotiation.cc:299] Negotiation successful
0929 03:02:57.283797 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6942,"thread_start_us":6794,"threads_started":1}
I20260929 03:02:57.283843 27652 tablet_service.cc:1467] Processing CreateTablet for tablet 0fcca99f6bd44f21b2538ddf38e34618 (DEFAULT_TABLE table=test-table [id=d80bdbd19a7d495b8dbc7c7fd35375fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:02:57.284137 27518 tablet_service.cc:1467] Processing CreateTablet for tablet 0fcca99f6bd44f21b2538ddf38e34618 (DEFAULT_TABLE table=test-table [id=d80bdbd19a7d495b8dbc7c7fd35375fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:02:57.285463 27518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fcca99f6bd44f21b2538ddf38e34618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:57.287909 27384 tablet_service.cc:1467] Processing CreateTablet for tablet 0fcca99f6bd44f21b2538ddf38e34618 (DEFAULT_TABLE table=test-table [id=d80bdbd19a7d495b8dbc7c7fd35375fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:02:57.288640 27652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fcca99f6bd44f21b2538ddf38e34618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:57.289356 27384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fcca99f6bd44f21b2538ddf38e34618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:57.300321 27735 tablet_bootstrap.cc:492] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64: Bootstrap starting.
I20260929 03:02:57.302209 27736 tablet_bootstrap.cc:492] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503: Bootstrap starting.
I20260929 03:02:57.303874 27737 tablet_bootstrap.cc:492] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273: Bootstrap starting.
I20260929 03:02:57.305513 27735 tablet_bootstrap.cc:654] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:57.305835 27736 tablet_bootstrap.cc:654] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:57.306824 27736 log.cc:824] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:57.307062 27735 log.cc:824] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:57.307665 27737 tablet_bootstrap.cc:654] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:57.309249 27737 log.cc:824] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:57.309790 27736 tablet_bootstrap.cc:492] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503: No bootstrap required, opened a new log
I20260929 03:02:57.310117 27736 ts_tablet_manager.cc:1397] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.002s
I20260929 03:02:57.311484 27735 tablet_bootstrap.cc:492] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64: No bootstrap required, opened a new log
I20260929 03:02:57.311793 27735 ts_tablet_manager.cc:1397] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260929 03:02:57.313943 27737 tablet_bootstrap.cc:492] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273: No bootstrap required, opened a new log
I20260929 03:02:57.314186 27737 ts_tablet_manager.cc:1397] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260929 03:02:57.319931 27736 raft_consensus.cc:348] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29670104c9f4cf6bf7f1aef54783503" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "5f6bd69b7f274dee845d943b6654fd64" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46021 } } peers { permanent_uuid: "440f7b035d0a4a9ea124d432de417273" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35593 } }
I20260929 03:02:57.320109 27735 raft_consensus.cc:348] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29670104c9f4cf6bf7f1aef54783503" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "5f6bd69b7f274dee845d943b6654fd64" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46021 } } peers { permanent_uuid: "440f7b035d0a4a9ea124d432de417273" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35593 } }
I20260929 03:02:57.320389 27736 raft_consensus.cc:374] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:57.320482 27736 raft_consensus.cc:729] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f29670104c9f4cf6bf7f1aef54783503, State: Initialized, Role: FOLLOWER
I20260929 03:02:57.320510 27735 raft_consensus.cc:374] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:57.320642 27735 raft_consensus.cc:729] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f6bd69b7f274dee845d943b6654fd64, State: Initialized, Role: FOLLOWER
I20260929 03:02:57.321229 27736 consensus_queue.cc:260] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29670104c9f4cf6bf7f1aef54783503" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "5f6bd69b7f274dee845d943b6654fd64" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46021 } } peers { permanent_uuid: "440f7b035d0a4a9ea124d432de417273" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35593 } }
I20260929 03:02:57.321323 27735 consensus_queue.cc:260] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29670104c9f4cf6bf7f1aef54783503" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "5f6bd69b7f274dee845d943b6654fd64" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46021 } } peers { permanent_uuid: "440f7b035d0a4a9ea124d432de417273" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35593 } }
I20260929 03:02:57.322552 27717 heartbeater.cc:503] Master 127.8.208.126:46727 was elected leader, sending a full tablet report...
I20260929 03:02:57.323037 27583 heartbeater.cc:503] Master 127.8.208.126:46727 was elected leader, sending a full tablet report...
I20260929 03:02:57.324271 27735 ts_tablet_manager.cc:1428] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260929 03:02:57.324545 27736 ts_tablet_manager.cc:1428] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503: Time spent starting tablet: real 0.014s user 0.006s sys 0.006s
I20260929 03:02:57.324357 27737 raft_consensus.cc:348] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29670104c9f4cf6bf7f1aef54783503" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "5f6bd69b7f274dee845d943b6654fd64" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46021 } } peers { permanent_uuid: "440f7b035d0a4a9ea124d432de417273" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35593 } }
I20260929 03:02:57.324786 27737 raft_consensus.cc:374] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:57.324882 27737 raft_consensus.cc:729] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 440f7b035d0a4a9ea124d432de417273, State: Initialized, Role: FOLLOWER
I20260929 03:02:57.325413 27737 consensus_queue.cc:260] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29670104c9f4cf6bf7f1aef54783503" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "5f6bd69b7f274dee845d943b6654fd64" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46021 } } peers { permanent_uuid: "440f7b035d0a4a9ea124d432de417273" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35593 } }
I20260929 03:02:57.326942 27737 ts_tablet_manager.cc:1428] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260929 03:02:57.380157 27741 raft_consensus.cc:482] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:02:57.380455 27741 raft_consensus.cc:504] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29670104c9f4cf6bf7f1aef54783503" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "5f6bd69b7f274dee845d943b6654fd64" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46021 } } peers { permanent_uuid: "440f7b035d0a4a9ea124d432de417273" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35593 } }
I20260929 03:02:57.382216 27741 leader_election.cc:288] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f6bd69b7f274dee845d943b6654fd64 (127.8.208.66:46021), 440f7b035d0a4a9ea124d432de417273 (127.8.208.65:35593)
W20260929 03:02:57.386709 27450 tablet.cc:2369] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:02:57.389328 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.382676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:46021 (local address 127.8.208.67:58469)
0929 03:02:57.383059 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.383106 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:57.383261 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:57.383712 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:57.383719 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:57.383755 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:57.383998 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:57.384010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.385268 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.385272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:57.387192 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:57.387205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.388567 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.388572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:57.388755 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.388790 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 03:02:57.388896 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 03:02:57.389021 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188}
I20260929 03:02:57.389359 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.382698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:58469 (local address 127.8.208.66:46021)
0929 03:02:57.383024 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:57.383031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:57.383209 (+ 178us) server_negotiation.cc:407] Connection header received
0929 03:02:57.383448 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:57.383457 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:57.383570 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:57.383662 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:57.384148 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.385084 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.387358 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.388394 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.388556 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.388626 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:57.388980 (+ 354us) server_negotiation.cc:299] Negotiation successful
0929 03:02:57.389099 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260929 03:02:57.390287 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.383311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:45235 (local address 127.8.208.65:35593)
0929 03:02:57.383533 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:57.383541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:57.384011 (+ 470us) server_negotiation.cc:407] Connection header received
0929 03:02:57.384159 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:57.384164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:57.384242 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:57.384332 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:02:57.384867 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.385708 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.387982 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.389217 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.389374 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.389435 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:57.389963 (+ 528us) server_negotiation.cc:299] Negotiation successful
0929 03:02:57.390099 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 03:02:57.390403 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.383101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35593 (local address 127.8.208.67:45235)
0929 03:02:57.383897 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.383914 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:57.384031 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:57.384380 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:57.384383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:57.384413 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:02:57.384708 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:57.384714 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.385848 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.385853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:57.387789 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:57.387801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.389623 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.389631 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:57.389783 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.389813 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 03:02:57.389908 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 03:02:57.390031 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":4,"thread_start_us":422,"threads_started":1}
I20260929 03:02:57.390343 27538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcca99f6bd44f21b2538ddf38e34618" candidate_uuid: "f29670104c9f4cf6bf7f1aef54783503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6bd69b7f274dee845d943b6654fd64" is_pre_election: true
I20260929 03:02:57.390864 27538 raft_consensus.cc:2454] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f29670104c9f4cf6bf7f1aef54783503 in term 0.
I20260929 03:02:57.391333 27404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcca99f6bd44f21b2538ddf38e34618" candidate_uuid: "f29670104c9f4cf6bf7f1aef54783503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "440f7b035d0a4a9ea124d432de417273" is_pre_election: true
I20260929 03:02:57.391469 27606 leader_election.cc:302] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f6bd69b7f274dee845d943b6654fd64, f29670104c9f4cf6bf7f1aef54783503; no voters:
I20260929 03:02:57.391819 27741 raft_consensus.cc:2781] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:02:57.391834 27404 raft_consensus.cc:2454] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f29670104c9f4cf6bf7f1aef54783503 in term 0.
I20260929 03:02:57.391978 27741 raft_consensus.cc:482] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:02:57.392096 27741 raft_consensus.cc:3037] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:57.394573 27741 raft_consensus.cc:504] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29670104c9f4cf6bf7f1aef54783503" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "5f6bd69b7f274dee845d943b6654fd64" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46021 } } peers { permanent_uuid: "440f7b035d0a4a9ea124d432de417273" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35593 } }
I20260929 03:02:57.395116 27741 leader_election.cc:288] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [CANDIDATE]: Term 1 election: Requested vote from peers 5f6bd69b7f274dee845d943b6654fd64 (127.8.208.66:46021), 440f7b035d0a4a9ea124d432de417273 (127.8.208.65:35593)
I20260929 03:02:57.395576 27538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcca99f6bd44f21b2538ddf38e34618" candidate_uuid: "f29670104c9f4cf6bf7f1aef54783503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6bd69b7f274dee845d943b6654fd64"
I20260929 03:02:57.395668 27404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcca99f6bd44f21b2538ddf38e34618" candidate_uuid: "f29670104c9f4cf6bf7f1aef54783503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "440f7b035d0a4a9ea124d432de417273"
I20260929 03:02:57.395790 27538 raft_consensus.cc:3037] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:57.395916 27404 raft_consensus.cc:3037] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:57.398116 27404 raft_consensus.cc:2454] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f29670104c9f4cf6bf7f1aef54783503 in term 1.
I20260929 03:02:57.398368 27538 raft_consensus.cc:2454] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f29670104c9f4cf6bf7f1aef54783503 in term 1.
I20260929 03:02:57.398521 27606 leader_election.cc:302] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 440f7b035d0a4a9ea124d432de417273, f29670104c9f4cf6bf7f1aef54783503; no voters:
I20260929 03:02:57.398874 27741 raft_consensus.cc:2781] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:02:57.399317 27741 raft_consensus.cc:686] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 1 LEADER]: Becoming Leader. State: Replica: f29670104c9f4cf6bf7f1aef54783503, State: Running, Role: LEADER
I20260929 03:02:57.399767 27741 consensus_queue.cc:237] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29670104c9f4cf6bf7f1aef54783503" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } } peers { permanent_uuid: "5f6bd69b7f274dee845d943b6654fd64" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46021 } } peers { permanent_uuid: "440f7b035d0a4a9ea124d432de417273" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35593 } }
I20260929 03:02:57.404103 27258 catalog_manager.cc:5495] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 reported cstate change: term changed from 0 to 1, leader changed from <none> to f29670104c9f4cf6bf7f1aef54783503 (127.8.208.67). New cstate: current_term: 1 leader_uuid: "f29670104c9f4cf6bf7f1aef54783503" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29670104c9f4cf6bf7f1aef54783503" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 42343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f6bd69b7f274dee845d943b6654fd64" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "440f7b035d0a4a9ea124d432de417273" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35593 } health_report { overall_health: UNKNOWN } } }
W20260929 03:02:57.421079 27584 tablet.cc:2369] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:02:57.432034 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.426400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35238 (local address 127.8.208.67:42343)
0929 03:02:57.426630 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:57.426636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:57.426657 (+ 21us) server_negotiation.cc:407] Connection header received
0929 03:02:57.426862 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:57.426866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:57.426941 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:57.427100 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:02:57.427477 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.428200 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.429769 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.430178 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.430321 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.430878 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:02:57.431355 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:02:57.431533 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:57.431680 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 03:02:57.431727 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260929 03:02:57.437747 27718 tablet.cc:2369] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:02:57.440548 27404 raft_consensus.cc:1261] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273 [term 1 FOLLOWER]: Refusing update from remote peer f29670104c9f4cf6bf7f1aef54783503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:02:57.440601 27538 raft_consensus.cc:1261] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64 [term 1 FOLLOWER]: Refusing update from remote peer f29670104c9f4cf6bf7f1aef54783503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:02:57.441527 27745 consensus_queue.cc:1035] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "440f7b035d0a4a9ea124d432de417273" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 35593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:02:57.441972 27741 consensus_queue.cc:1035] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f6bd69b7f274dee845d943b6654fd64" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 46021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:02:57.454209 27748 mvcc.cc:205] Tried to move back new op lower bound from 7334506403586166784 to 7334506403432996864. Current Snapshot: MvccSnapshot[applied={T|T < 7334506403586166784}]
I20260929 03:02:57.466958 27259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:02:57.467226 27259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:02:57.470870 27259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92d2034650fc44d195f3081f2a2bff95: Sending DeleteTablet for 3 replicas of tablet 0fcca99f6bd44f21b2538ddf38e34618
I20260929 03:02:57.472023 27652 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fcca99f6bd44f21b2538ddf38e34618 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:02:57 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:44192
I20260929 03:02:57.472074 27518 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fcca99f6bd44f21b2538ddf38e34618 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:02:57 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:56310
I20260929 03:02:57.472172 27384 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fcca99f6bd44f21b2538ddf38e34618 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:02:57 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:51994
I20260929 03:02:57.479828 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 27321
I20260929 03:02:57.485581 27758 tablet_replica.cc:330] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503: stopping tablet replica
I20260929 03:02:57.486101 27758 raft_consensus.cc:2229] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:02:57.486843 27758 raft_consensus.cc:2258] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:02:57.499517 27757 tablet_replica.cc:330] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64: stopping tablet replica
I20260929 03:02:57.499985 27757 raft_consensus.cc:2229] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:02:57.500583 27757 raft_consensus.cc:2258] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:02:57.502753 27757 ts_tablet_manager.cc:1905] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:02:57.507637 27759 tablet_replica.cc:330] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273: stopping tablet replica
I20260929 03:02:57.508062 27759 raft_consensus.cc:2229] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:02:57.508553 27759 raft_consensus.cc:2258] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:02:57.509637 27758 ts_tablet_manager.cc:1905] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 03:02:57 dist-test-slave-m1rv krb5kdc[27215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 03:02:57.512279 27757 ts_tablet_manager.cc:1918] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:02:57.512452 27757 log.cc:1196] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/0fcca99f6bd44f21b2538ddf38e34618
I20260929 03:02:57.512984 27757 ts_tablet_manager.cc:1939] T 0fcca99f6bd44f21b2538ddf38e34618 P 5f6bd69b7f274dee845d943b6654fd64: Deleting consensus metadata
I20260929 03:02:57.514883 27245 catalog_manager.cc:4841] TS 5f6bd69b7f274dee845d943b6654fd64 (127.8.208.66:46021): tablet 0fcca99f6bd44f21b2538ddf38e34618 (table test-table [id=d80bdbd19a7d495b8dbc7c7fd35375fe]) successfully deleted
I20260929 03:02:57.517081 27759 ts_tablet_manager.cc:1905] T 0fcca99f6bd44f21b2538ddf38e34618 P 440f7b035d0a4a9ea124d432de417273: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:02:57.517316 27758 ts_tablet_manager.cc:1918] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:02:57.517496 27758 log.cc:1196] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/0fcca99f6bd44f21b2538ddf38e34618
I20260929 03:02:57.518066 27758 ts_tablet_manager.cc:1939] T 0fcca99f6bd44f21b2538ddf38e34618 P f29670104c9f4cf6bf7f1aef54783503: Deleting consensus metadata
I20260929 03:02:57.519785 27247 catalog_manager.cc:4841] TS f29670104c9f4cf6bf7f1aef54783503 (127.8.208.67:42343): tablet 0fcca99f6bd44f21b2538ddf38e34618 (table test-table [id=d80bdbd19a7d495b8dbc7c7fd35375fe]) successfully deleted
I20260929 03:02:57.521471 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.481611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36724 (local address 127.8.208.65:35593)
0929 03:02:57.481805 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:57.481812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:57.499762 (+ 17950us) server_negotiation.cc:407] Connection header received
0929 03:02:57.500298 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:57.500304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:57.500392 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:57.500469 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:57.502524 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.503674 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.505621 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.506319 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.506414 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.513481 (+ 7067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:57.513500 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:57.513517 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:57.513569 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:57.517677 (+ 4108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:57.518638 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:57.518646 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:57.518649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:57.518841 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:57.519319 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:57.519324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:57.519327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:57.519796 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:57.520147 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:57.520639 (+ 492us) server_negotiation.cc:299] Negotiation successful
0929 03:02:57.520887 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 03:02:57.522132 27444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:57.614284 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 27321
W20260929 03:02:57.633384 27245 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.208.65:35593 (error 108)
I20260929 03:02:57.634943 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 27455
I20260929 03:02:57.636214 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.635272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35593 (local address 127.0.0.1:36736)
0929 03:02:57.635714 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.635948 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:35593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
W20260929 03:02:57.636472 27245 catalog_manager.cc:4584] TS 440f7b035d0a4a9ea124d432de417273 (127.8.208.65:35593): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0fcca99f6bd44f21b2538ddf38e34618: Network error: Client connection negotiation failed: client connection to 127.8.208.65:35593: connect: Connection refused (error 111)
Sep 29 03:02:57 dist-test-slave-m1rv krb5kdc[27215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 03:02:57.651620 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.636170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43448 (local address 127.8.208.66:46021)
0929 03:02:57.636366 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:57.636372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:57.636392 (+ 20us) server_negotiation.cc:407] Connection header received
0929 03:02:57.636459 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:57.636465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:57.636546 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:57.636659 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:57.637998 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.639098 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.640334 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.641010 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.641154 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.645952 (+ 4798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:57.645966 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:57.645980 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:57.646012 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:57.648734 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:57.649565 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:57.649572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:57.649575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:57.649694 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:57.650095 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:57.650098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:57.650101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:57.650578 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:57.650796 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:57.651159 (+ 363us) server_negotiation.cc:299] Negotiation successful
0929 03:02:57.651319 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 03:02:57.652436 27578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:57.690449 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.689911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35593 (local address 127.0.0.1:36738)
0929 03:02:57.690155 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.690259 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:35593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260929 03:02:57.691670 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.691231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35593 (local address 127.0.0.1:36748)
0929 03:02:57.691403 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.691496 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:35593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 03:02:57.742028 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 27455
I20260929 03:02:57.752056 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.751605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35593 (local address 127.0.0.1:36756)
0929 03:02:57.751808 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.751891 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:35593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 03:02:57.753108 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.752766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35593 (local address 127.0.0.1:36764)
0929 03:02:57.752914 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.753003 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:35593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 03:02:57.765610 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 27589
Sep 29 03:02:57 dist-test-slave-m1rv krb5kdc[27215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 03:02:57.781855 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.766460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.8.208.67:42343)
0929 03:02:57.766688 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:57.766694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:57.766714 (+ 20us) server_negotiation.cc:407] Connection header received
0929 03:02:57.766815 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:57.766821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:57.766903 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:57.767019 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:57.768347 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.769251 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.770537 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:57.770980 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:57.771129 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:57.775959 (+ 4830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:57.775970 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:57.775985 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:02:57.776020 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:57.779007 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:57.779671 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:57.779678 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:57.779680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:57.779887 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:02:57.780370 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:02:57.780374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:02:57.780376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:02:57.780736 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:57.780973 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:57.781355 (+ 382us) server_negotiation.cc:299] Negotiation successful
0929 03:02:57.781534 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260929 03:02:57.782549 27712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:57.833947 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.833508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35593 (local address 127.0.0.1:36774)
0929 03:02:57.833704 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.833795 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:35593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 03:02:57.834992 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:57.834574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:35593 (local address 127.0.0.1:36778)
0929 03:02:57.834804 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:57.834890 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.65:35593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260929 03:02:57.908811 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 27589
I20260929 03:02:57.928272 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 27228
I20260929 03:02:57.929479 27290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:02:58.041378 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 27228
2026-09-29T03:02:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4342 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T03:02:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:02:58Z Disabled control of system clock
I20260929 03:02:58.117672 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:39317
--webserver_interface=127.8.208.126
--webserver_port=0
--builtin_ntp_servers=127.8.208.84:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:39317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 03:02:58.438617 27769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:58.438889 27769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:58.438983 27769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:58.446096 27769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:02:58.446192 27769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:58.446235 27769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:58.446291 27769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:02:58.446333 27769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:02:58.455287 27769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:39317
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:39317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:58.456465 27769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:58.458237 27769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:58.470158 27774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:58.470191 27777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:58.470217 27775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:58.471067 27769 server_base.cc:1034] running on GCE node
I20260929 03:02:58.471822 27769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:58.473326 27769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:58.474579 27769 hybrid_clock.cc:648] HybridClock initialized: now 1790650978474530 us; error 61 us; skew 500 ppm
I20260929 03:02:58.475031 27769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 03:02:58.485251 27769 webserver.cc:460] Webserver started at http://127.8.208.126:36667/ using document root <none> and password file <none>
I20260929 03:02:58.486022 27769 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:58.486162 27769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:58.486485 27769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:58.489709 27769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "c452b02f370b45b290f3d326502d3d21"
format_stamp: "Formatted at 2026-09-29 03:02:58 on dist-test-slave-m1rv"
server_key: "6c346440cc5b257728880a20ca1adb8c"
server_key_iv: "5d197251dd366f66c9951bc35213336f"
server_key_version: "encryptionkey@0"
I20260929 03:02:58.490527 27769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "c452b02f370b45b290f3d326502d3d21"
format_stamp: "Formatted at 2026-09-29 03:02:58 on dist-test-slave-m1rv"
server_key: "6c346440cc5b257728880a20ca1adb8c"
server_key_iv: "5d197251dd366f66c9951bc35213336f"
server_key_version: "encryptionkey@0"
I20260929 03:02:58.495708 27769 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 03:02:58.499293 27783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:58.500682 27769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:02:58.500893 27769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "c452b02f370b45b290f3d326502d3d21"
format_stamp: "Formatted at 2026-09-29 03:02:58 on dist-test-slave-m1rv"
server_key: "6c346440cc5b257728880a20ca1adb8c"
server_key_iv: "5d197251dd366f66c9951bc35213336f"
server_key_version: "encryptionkey@0"
I20260929 03:02:58.501176 27769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:58.519989 27769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:58.520812 27769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:58.521109 27769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:58.539103 27769 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:39317
I20260929 03:02:58.539132 27834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:39317 every 8 connection(s)
I20260929 03:02:58.540442 27769 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:02:58.544489 27835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:02:58.549803 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 27769
I20260929 03:02:58.549979 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:02:58.550465 9025 external_mini_cluster.cc:1428] Setting key 461e4e6ae6710f5d02a2200ae030f1a6
I20260929 03:02:58.555054 27835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21: Bootstrap starting.
I20260929 03:02:58.558475 27835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21: Neither blocks nor log segments found. Creating new log.
I20260929 03:02:58.559664 27835 log.cc:824] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21: Log is configured to *not* fsync() on all Append() calls
I20260929 03:02:58.563138 27835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21: No bootstrap required, opened a new log
I20260929 03:02:58.564821 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:58.552363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45918 (local address 127.8.208.126:39317)
0929 03:02:58.553494 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:58.553505 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:58.553544 (+ 39us) server_negotiation.cc:407] Connection header received
0929 03:02:58.555225 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:58.555259 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:58.555680 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:58.555997 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:58.556596 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:58.557942 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:58.559920 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:58.560611 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:58.560748 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:58.562072 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:58.562081 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:58.562098 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:02:58.562788 (+ 690us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:58.562900 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:02:58.562925 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:58.563042 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:58.563254 (+ 212us) server_negotiation.cc:299] Negotiation successful
0929 03:02:58.563584 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":420,"threads_started":1}
I20260929 03:02:58.571571 27835 raft_consensus.cc:348] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c452b02f370b45b290f3d326502d3d21" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39317 } }
I20260929 03:02:58.571875 27835 raft_consensus.cc:374] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:02:58.571959 27835 raft_consensus.cc:729] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c452b02f370b45b290f3d326502d3d21, State: Initialized, Role: FOLLOWER
I20260929 03:02:58.572397 27835 consensus_queue.cc:260] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c452b02f370b45b290f3d326502d3d21" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39317 } }
I20260929 03:02:58.572592 27835 raft_consensus.cc:388] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:02:58.572681 27835 raft_consensus.cc:482] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:02:58.572801 27835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:02:58.574935 27835 raft_consensus.cc:504] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c452b02f370b45b290f3d326502d3d21" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39317 } }
I20260929 03:02:58.575362 27835 leader_election.cc:302] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c452b02f370b45b290f3d326502d3d21; no voters:
I20260929 03:02:58.575879 27835 leader_election.cc:288] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:02:58.576161 27840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:02:58.577302 27840 raft_consensus.cc:686] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [term 1 LEADER]: Becoming Leader. State: Replica: c452b02f370b45b290f3d326502d3d21, State: Running, Role: LEADER
I20260929 03:02:58.577764 27840 consensus_queue.cc:237] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c452b02f370b45b290f3d326502d3d21" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39317 } }
I20260929 03:02:58.578119 27835 sys_catalog.cc:564] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:02:58.580485 27842 sys_catalog.cc:455] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c452b02f370b45b290f3d326502d3d21. Latest consensus state: current_term: 1 leader_uuid: "c452b02f370b45b290f3d326502d3d21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c452b02f370b45b290f3d326502d3d21" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39317 } } }
I20260929 03:02:58.580569 27841 sys_catalog.cc:455] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c452b02f370b45b290f3d326502d3d21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c452b02f370b45b290f3d326502d3d21" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 39317 } } }
I20260929 03:02:58.580860 27842 sys_catalog.cc:458] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [sys.catalog]: This master's current role is: LEADER
I20260929 03:02:58.580976 27841 sys_catalog.cc:458] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21 [sys.catalog]: This master's current role is: LEADER
I20260929 03:02:58.586153 27847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:02:58.591478 27847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:02:58.599972 27847 catalog_manager.cc:1347] Generated new cluster ID: dbf4ffbd456948f7950a748140e6a334
I20260929 03:02:58.600091 27847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:02:58.622524 27847 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:02:58.623736 27847 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:02:58.636502 27847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21: Generated new TSK 0
I20260929 03:02:58.637362 27847 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:02:58.648042 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39317
--builtin_ntp_servers=127.8.208.84:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 03:02:58.984747 27859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:58.985044 27859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:58.985138 27859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:58.994324 27859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:58.994424 27859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:58.994535 27859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:02:59.004734 27859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:59.006167 27859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:59.007884 27859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:59.020634 27864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:59.020686 27867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:59.020709 27865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:59.021613 27859 server_base.cc:1034] running on GCE node
I20260929 03:02:59.022174 27859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:59.023046 27859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:59.024248 27859 hybrid_clock.cc:648] HybridClock initialized: now 1790650979024204 us; error 43 us; skew 500 ppm
I20260929 03:02:59.024564 27859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 03:02:59.026877 27859 webserver.cc:460] Webserver started at http://127.8.208.65:40579/ using document root <none> and password file <none>
I20260929 03:02:59.027555 27859 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:59.027688 27859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:59.028012 27859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:59.030853 27859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "18e2728c01a54512a3025cf287dd0f30"
format_stamp: "Formatted at 2026-09-29 03:02:59 on dist-test-slave-m1rv"
server_key: "484404360acba605d3489b6c878813e7"
server_key_iv: "2a3f5dade1595b6ce549ed2a7d05062b"
server_key_version: "encryptionkey@0"
I20260929 03:02:59.031623 27859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "18e2728c01a54512a3025cf287dd0f30"
format_stamp: "Formatted at 2026-09-29 03:02:59 on dist-test-slave-m1rv"
server_key: "484404360acba605d3489b6c878813e7"
server_key_iv: "2a3f5dade1595b6ce549ed2a7d05062b"
server_key_version: "encryptionkey@0"
I20260929 03:02:59.036538 27859 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 03:02:59.039779 27873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:59.041062 27859 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 03:02:59.041242 27859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "18e2728c01a54512a3025cf287dd0f30"
format_stamp: "Formatted at 2026-09-29 03:02:59 on dist-test-slave-m1rv"
server_key: "484404360acba605d3489b6c878813e7"
server_key_iv: "2a3f5dade1595b6ce549ed2a7d05062b"
server_key_version: "encryptionkey@0"
I20260929 03:02:59.041453 27859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:59.058661 27859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:59.059448 27859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:59.059752 27859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:59.060724 27859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:59.062145 27859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:59.062268 27859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:59.062422 27859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:59.062513 27859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:59.102720 27859 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:34665
I20260929 03:02:59.102730 27985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:34665 every 8 connection(s)
I20260929 03:02:59.104512 27859 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:02:59.111734 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 27859
I20260929 03:02:59.111917 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 03:02:59.112449 9025 external_mini_cluster.cc:1428] Setting key 626e2e1c20e18c2ff962b146ada239cd
I20260929 03:02:59.115077 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39317
--builtin_ntp_servers=127.8.208.84:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 03:02:59.120326 27988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:59.107334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:36911 (local address 127.8.208.126:39317)
0929 03:02:59.107826 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:59.107832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:59.109208 (+ 1376us) server_negotiation.cc:407] Connection header received
0929 03:02:59.111041 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:59.111047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:59.111143 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:59.111276 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:59.112711 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:59.113989 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:59.116661 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:59.117417 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:59.117546 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:59.119404 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:59.119410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:59.119414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:02:59.119436 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:59.119491 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:02:59.119506 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:59.119574 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:59.119894 (+ 320us) server_negotiation.cc:299] Negotiation successful
0929 03:02:59.119991 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":241,"threads_started":1}
I20260929 03:02:59.122304 27989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:59.107981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39317 (local address 127.8.208.65:36911)
0929 03:02:59.109013 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:59.109057 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:59.110772 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:59.111575 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:59.111589 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:59.111708 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:59.112517 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:59.112542 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:59.114185 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:59.114190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:59.116439 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:59.116456 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:59.117884 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:59.117892 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:59.118232 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:59.119044 (+ 812us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:02:59.119047 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:59.119101 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:02:59.119198 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:59.119659 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:59.119670 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:59.119677 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 03:02:59.120599 (+ 922us) client_negotiation.cc:240] Negotiation successful
0929 03:02:59.121003 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":343,"threads_started":1}
I20260929 03:02:59.124197 27986 heartbeater.cc:347] Connected to a master server at 127.8.208.126:39317
I20260929 03:02:59.124492 27986 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:59.125413 27986 heartbeater.cc:511] Master 127.8.208.126:39317 requested a full tablet report, sending...
I20260929 03:02:59.127727 27800 ts_manager.cc:194] Registered new tserver with Master: 18e2728c01a54512a3025cf287dd0f30 (127.8.208.65:34665)
I20260929 03:02:59.129904 27800 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.208.65:36911
W20260929 03:02:59.440765 27990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:59.441090 27990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:59.441324 27990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:59.448827 27990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:59.448957 27990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:59.449141 27990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:02:59.458160 27990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:59.459424 27990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:59.461251 27990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:59.473917 27998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:59.474036 27995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:59.474007 27996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:59.478820 27990 server_base.cc:1034] running on GCE node
I20260929 03:02:59.479467 27990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:59.480391 27990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:59.481627 27990 hybrid_clock.cc:648] HybridClock initialized: now 1790650979481595 us; error 54 us; skew 500 ppm
I20260929 03:02:59.481941 27990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 03:02:59.484169 27990 webserver.cc:460] Webserver started at http://127.8.208.66:45771/ using document root <none> and password file <none>
I20260929 03:02:59.484864 27990 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:59.484997 27990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:59.485386 27990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:59.488227 27990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "dfd387fd9dc0445182b207e5f80ba735"
format_stamp: "Formatted at 2026-09-29 03:02:59 on dist-test-slave-m1rv"
server_key: "f8dbe8f2e5407c6089c73a4c8ad7442c"
server_key_iv: "c3b6d48269dba0729a67cf6eab789d26"
server_key_version: "encryptionkey@0"
I20260929 03:02:59.489012 27990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "dfd387fd9dc0445182b207e5f80ba735"
format_stamp: "Formatted at 2026-09-29 03:02:59 on dist-test-slave-m1rv"
server_key: "f8dbe8f2e5407c6089c73a4c8ad7442c"
server_key_iv: "c3b6d48269dba0729a67cf6eab789d26"
server_key_version: "encryptionkey@0"
I20260929 03:02:59.494230 27990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260929 03:02:59.497951 28004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:59.499290 27990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:02:59.499504 27990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "dfd387fd9dc0445182b207e5f80ba735"
format_stamp: "Formatted at 2026-09-29 03:02:59 on dist-test-slave-m1rv"
server_key: "f8dbe8f2e5407c6089c73a4c8ad7442c"
server_key_iv: "c3b6d48269dba0729a67cf6eab789d26"
server_key_version: "encryptionkey@0"
I20260929 03:02:59.499773 27990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:59.526340 27990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:59.527235 27990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:59.527577 27990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:59.528549 27990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:59.530043 27990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:59.530181 27990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:59.530305 27990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:59.530396 27990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:59.564041 27990 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:41901
I20260929 03:02:59.564127 28116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:41901 every 8 connection(s)
I20260929 03:02:59.565335 27990 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:02:59.573223 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 27990
I20260929 03:02:59.573405 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 03:02:59.573853 9025 external_mini_cluster.cc:1428] Setting key d2f1c2d8cf6a564aa3ed1066a0fd6e06
I20260929 03:02:59.576524 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39317
--builtin_ntp_servers=127.8.208.84:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 03:02:59.578471 27988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:59.568079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:58883 (local address 127.8.208.126:39317)
0929 03:02:59.568275 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 03:02:59.568280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:02:59.569645 (+ 1365us) server_negotiation.cc:407] Connection header received
0929 03:02:59.571520 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:02:59.571527 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:02:59.571620 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:02:59.571729 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:02:59.572777 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:59.573674 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:59.575576 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:02:59.575990 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:02:59.576120 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:59.577569 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:02:59.577574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:02:59.577577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:02:59.577593 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:02:59.577625 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:02:59.577636 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:02:59.577725 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 03:02:59.578065 (+ 340us) server_negotiation.cc:299] Negotiation successful
0929 03:02:59.578157 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 03:02:59.579522 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:02:59.568494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39317 (local address 127.8.208.66:58883)
0929 03:02:59.569476 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0929 03:02:59.569512 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 03:02:59.571247 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:02:59.571960 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:02:59.571968 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:02:59.572077 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:02:59.572609 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:59.572625 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:59.573819 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:59.573830 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:59.575344 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:02:59.575384 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:02:59.576218 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:02:59.576226 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:02:59.576454 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:02:59.577259 (+ 805us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:02:59.577262 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:02:59.577300 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:02:59.577378 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:02:59.577761 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:02:59.577776 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:02:59.577785 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 03:02:59.578057 (+ 272us) client_negotiation.cc:240] Negotiation successful
0929 03:02:59.578438 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":273,"threads_started":1}
I20260929 03:02:59.580967 28117 heartbeater.cc:347] Connected to a master server at 127.8.208.126:39317
I20260929 03:02:59.581346 28117 heartbeater.cc:464] Registering TS with master...
I20260929 03:02:59.582204 28117 heartbeater.cc:511] Master 127.8.208.126:39317 requested a full tablet report, sending...
I20260929 03:02:59.583681 27800 ts_manager.cc:194] Registered new tserver with Master: dfd387fd9dc0445182b207e5f80ba735 (127.8.208.66:41901)
I20260929 03:02:59.584666 27800 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.208.66:58883
W20260929 03:02:59.900838 28120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:02:59.901100 28120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:02:59.901258 28120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:02:59.908573 28120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:02:59.908668 28120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:02:59.908869 28120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:02:59.917743 28120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:39317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:02:59.918990 28120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:02:59.920763 28120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:02:59.933768 28125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:59.933706 28126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:02:59.933735 28128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:02:59.935025 28120 server_base.cc:1034] running on GCE node
I20260929 03:02:59.935624 28120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:02:59.936527 28120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:02:59.937780 28120 hybrid_clock.cc:648] HybridClock initialized: now 1790650979937729 us; error 52 us; skew 500 ppm
I20260929 03:02:59.938100 28120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 03:02:59.940419 28120 webserver.cc:460] Webserver started at http://127.8.208.67:39557/ using document root <none> and password file <none>
I20260929 03:02:59.941084 28120 fs_manager.cc:359] Metadata directory not provided
I20260929 03:02:59.941234 28120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:02:59.941562 28120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:02:59.944311 28120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "c9afab83d1b9466bb94a219407479fd3"
format_stamp: "Formatted at 2026-09-29 03:02:59 on dist-test-slave-m1rv"
server_key: "abd3597f2768a10a7f3b9f5caee5215d"
server_key_iv: "af61080ecc3a9f2e3950e56074b24779"
server_key_version: "encryptionkey@0"
I20260929 03:02:59.945160 28120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "c9afab83d1b9466bb94a219407479fd3"
format_stamp: "Formatted at 2026-09-29 03:02:59 on dist-test-slave-m1rv"
server_key: "abd3597f2768a10a7f3b9f5caee5215d"
server_key_iv: "af61080ecc3a9f2e3950e56074b24779"
server_key_version: "encryptionkey@0"
I20260929 03:02:59.950222 28120 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260929 03:02:59.953619 28134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:59.954941 28120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260929 03:02:59.955121 28120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "c9afab83d1b9466bb94a219407479fd3"
format_stamp: "Formatted at 2026-09-29 03:02:59 on dist-test-slave-m1rv"
server_key: "abd3597f2768a10a7f3b9f5caee5215d"
server_key_iv: "af61080ecc3a9f2e3950e56074b24779"
server_key_version: "encryptionkey@0"
I20260929 03:02:59.955339 28120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:02:59.972831 28120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:02:59.973660 28120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:02:59.973969 28120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:02:59.974885 28120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:02:59.976261 28120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:02:59.976387 28120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:02:59.976518 28120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:02:59.976619 28120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:00.009840 28120 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:40007
I20260929 03:03:00.009951 28246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:40007 every 8 connection(s)
I20260929 03:03:00.011165 28120 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:03:00.011566 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 28120
I20260929 03:03:00.011754 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 03:03:00.012406 9025 external_mini_cluster.cc:1428] Setting key 81f973550d428b205511b57684cf0b77
I20260929 03:03:00.024154 27988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.013794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:45719 (local address 127.8.208.126:39317)
0929 03:03:00.013978 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:00.013982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:00.015698 (+ 1716us) server_negotiation.cc:407] Connection header received
0929 03:03:00.017866 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:00.017872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:00.017964 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:00.018070 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:00.019110 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.019843 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.021463 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.021873 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.022008 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.023327 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:00.023332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:00.023335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:00.023351 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:00.023387 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:00.023399 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:00.023497 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:00.023723 (+ 226us) server_negotiation.cc:299] Negotiation successful
0929 03:03:00.023816 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 03:03:00.025102 28249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.014459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:39317 (local address 127.8.208.67:45719)
0929 03:03:00.015554 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:00.015596 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:00.017619 (+ 2023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:00.018302 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:00.018311 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:00.018400 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:00.018950 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:00.018966 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.019999 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.020006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:00.021288 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:00.021322 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.022090 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.022098 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:00.022325 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.023011 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:03:00.023013 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:00.023052 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:03:00.023125 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:00.023498 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:00.023510 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:00.023520 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 03:03:00.023710 (+ 190us) client_negotiation.cc:240] Negotiation successful
0929 03:03:00.024107 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":300,"threads_started":1}
I20260929 03:03:00.026646 28247 heartbeater.cc:347] Connected to a master server at 127.8.208.126:39317
I20260929 03:03:00.026973 28247 heartbeater.cc:464] Registering TS with master...
I20260929 03:03:00.027782 28247 heartbeater.cc:511] Master 127.8.208.126:39317 requested a full tablet report, sending...
I20260929 03:03:00.029268 27800 ts_manager.cc:194] Registered new tserver with Master: c9afab83d1b9466bb94a219407479fd3 (127.8.208.67:40007)
I20260929 03:03:00.030138 27800 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.208.67:45719
I20260929 03:03:00.040221 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:03:00.050586 27988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.044298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.8.208.126:39317)
0929 03:03:00.044523 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:00.044534 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:00.044880 (+ 346us) server_negotiation.cc:407] Connection header received
0929 03:03:00.045049 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:00.045052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:00.045122 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:00.045312 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:00.045837 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.046531 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.047923 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.048325 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.048460 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.049847 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:00.049858 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:00.049861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:00.049878 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:00.049907 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:00.049919 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:00.050068 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:00.050262 (+ 194us) server_negotiation.cc:299] Negotiation successful
0929 03:03:00.050316 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 03:03:00.054303 27800 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:03:00.061291 27800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:03:00.099680 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.072827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:40007 (local address 127.0.0.1:56106)
0929 03:03:00.090019 (+ 17192us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:00.090066 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:00.090909 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:00.091924 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:00.091932 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:00.091973 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:00.092369 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:00.092381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.094450 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.094458 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:00.096767 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:00.096779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.098675 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.098681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:00.098902 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.098959 (+ 57us) client_negotiation.cc:769] Sending connection context
0929 03:03:00.099068 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 03:03:00.099269 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16898,"mutex_wait_us":1,"thread_start_us":16717,"threads_started":1}
I20260929 03:03:00.100299 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.073753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55418 (local address 127.8.208.66:41901)
0929 03:03:00.089296 (+ 15543us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:00.089309 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:00.091515 (+ 2206us) server_negotiation.cc:407] Connection header received
0929 03:03:00.091801 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:00.091808 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:00.092052 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:00.092219 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:00.093120 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.094568 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.097230 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.098678 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.098809 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.099059 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:00.099584 (+ 525us) server_negotiation.cc:299] Negotiation successful
0929 03:03:00.099775 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15303,"thread_start_us":15170,"threads_started":1}
I20260929 03:03:00.100776 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.073220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:41901 (local address 127.0.0.1:55418)
0929 03:03:00.091388 (+ 18168us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:00.091405 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:00.091618 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:00.092513 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:00.092518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:00.092546 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:00.092938 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:00.092949 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.094699 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.094707 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:00.097011 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:00.097024 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.099189 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.099195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:00.099366 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.099396 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 03:03:00.100271 (+ 875us) client_negotiation.cc:240] Negotiation successful
0929 03:03:00.100421 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18055,"mutex_wait_us":3,"thread_start_us":17941,"threads_started":1}
I20260929 03:03:00.100775 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.072926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56106 (local address 127.8.208.67:40007)
0929 03:03:00.088430 (+ 15504us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:00.088455 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:00.090205 (+ 1750us) server_negotiation.cc:407] Connection header received
0929 03:03:00.091097 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:00.091109 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:00.091416 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:00.091616 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:00.092596 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.094319 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.096970 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.098484 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.099908 (+ 1424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.100171 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:00.100239 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 03:03:00.100420 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15228,"thread_start_us":15555,"threads_started":1}
I20260929 03:03:00.100996 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.073564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50708 (local address 127.8.208.65:34665)
0929 03:03:00.093247 (+ 19683us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:00.093254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:00.093277 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:03:00.093349 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:00.093353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:00.093512 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:00.093715 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:00.094275 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.096229 (+ 1954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.098211 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.099455 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.099584 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.099847 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:00.100462 (+ 615us) server_negotiation.cc:299] Negotiation successful
0929 03:03:00.100621 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19475,"thread_start_us":19342,"threads_started":1}
I20260929 03:03:00.101357 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.072931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:34665 (local address 127.0.0.1:50708)
0929 03:03:00.090587 (+ 17656us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:00.090602 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:00.090744 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:00.093697 (+ 2953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:00.093701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:00.093725 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:00.094096 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:00.094105 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.096378 (+ 2273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.096384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:00.098033 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:00.098045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.099984 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.099991 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:00.100182 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.100217 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 03:03:00.100296 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 03:03:00.100999 (+ 703us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17558,"mutex_wait_us":575,"thread_start_us":17448,"threads_started":1}
I20260929 03:03:00.104857 28052 tablet_service.cc:1467] Processing CreateTablet for tablet fe9d682b80594e4fbddef59d9efff976 (DEFAULT_TABLE table=test-table [id=c4688adaeadf4895bb190fc83655d52c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:03:00.104954 27921 tablet_service.cc:1467] Processing CreateTablet for tablet fe9d682b80594e4fbddef59d9efff976 (DEFAULT_TABLE table=test-table [id=c4688adaeadf4895bb190fc83655d52c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:03:00.104867 28182 tablet_service.cc:1467] Processing CreateTablet for tablet fe9d682b80594e4fbddef59d9efff976 (DEFAULT_TABLE table=test-table [id=c4688adaeadf4895bb190fc83655d52c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:03:00.106276 28182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe9d682b80594e4fbddef59d9efff976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:00.106285 27921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe9d682b80594e4fbddef59d9efff976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:00.106272 28052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe9d682b80594e4fbddef59d9efff976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:00.119477 28265 tablet_bootstrap.cc:492] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735: Bootstrap starting.
I20260929 03:03:00.122452 28267 tablet_bootstrap.cc:492] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3: Bootstrap starting.
I20260929 03:03:00.123073 28265 tablet_bootstrap.cc:654] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:00.124126 28266 tablet_bootstrap.cc:492] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30: Bootstrap starting.
I20260929 03:03:00.124406 28265 log.cc:824] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:00.126398 28267 tablet_bootstrap.cc:654] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:00.127665 28267 log.cc:824] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:00.127848 28266 tablet_bootstrap.cc:654] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:00.128981 28266 log.cc:824] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:00.129308 28265 tablet_bootstrap.cc:492] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735: No bootstrap required, opened a new log
I20260929 03:03:00.129609 28265 ts_tablet_manager.cc:1397] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20260929 03:03:00.130570 28267 tablet_bootstrap.cc:492] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3: No bootstrap required, opened a new log
I20260929 03:03:00.130824 28267 ts_tablet_manager.cc:1397] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260929 03:03:00.131863 28266 tablet_bootstrap.cc:492] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30: No bootstrap required, opened a new log
I20260929 03:03:00.132160 28266 ts_tablet_manager.cc:1397] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260929 03:03:00.135072 27986 heartbeater.cc:503] Master 127.8.208.126:39317 was elected leader, sending a full tablet report...
I20260929 03:03:00.139845 28267 raft_consensus.cc:348] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9afab83d1b9466bb94a219407479fd3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40007 } } peers { permanent_uuid: "18e2728c01a54512a3025cf287dd0f30" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34665 } } peers { permanent_uuid: "dfd387fd9dc0445182b207e5f80ba735" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41901 } }
I20260929 03:03:00.139957 28265 raft_consensus.cc:348] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9afab83d1b9466bb94a219407479fd3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40007 } } peers { permanent_uuid: "18e2728c01a54512a3025cf287dd0f30" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34665 } } peers { permanent_uuid: "dfd387fd9dc0445182b207e5f80ba735" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41901 } }
I20260929 03:03:00.140228 28267 raft_consensus.cc:374] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:00.140350 28267 raft_consensus.cc:729] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9afab83d1b9466bb94a219407479fd3, State: Initialized, Role: FOLLOWER
I20260929 03:03:00.140369 28265 raft_consensus.cc:374] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:00.140470 28265 raft_consensus.cc:729] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfd387fd9dc0445182b207e5f80ba735, State: Initialized, Role: FOLLOWER
I20260929 03:03:00.141111 28267 consensus_queue.cc:260] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9afab83d1b9466bb94a219407479fd3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40007 } } peers { permanent_uuid: "18e2728c01a54512a3025cf287dd0f30" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34665 } } peers { permanent_uuid: "dfd387fd9dc0445182b207e5f80ba735" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41901 } }
I20260929 03:03:00.141296 28265 consensus_queue.cc:260] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9afab83d1b9466bb94a219407479fd3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40007 } } peers { permanent_uuid: "18e2728c01a54512a3025cf287dd0f30" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34665 } } peers { permanent_uuid: "dfd387fd9dc0445182b207e5f80ba735" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41901 } }
I20260929 03:03:00.143095 28266 raft_consensus.cc:348] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9afab83d1b9466bb94a219407479fd3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40007 } } peers { permanent_uuid: "18e2728c01a54512a3025cf287dd0f30" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34665 } } peers { permanent_uuid: "dfd387fd9dc0445182b207e5f80ba735" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41901 } }
I20260929 03:03:00.143622 28266 raft_consensus.cc:374] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:00.143736 28266 raft_consensus.cc:729] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e2728c01a54512a3025cf287dd0f30, State: Initialized, Role: FOLLOWER
I20260929 03:03:00.144428 28266 consensus_queue.cc:260] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9afab83d1b9466bb94a219407479fd3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40007 } } peers { permanent_uuid: "18e2728c01a54512a3025cf287dd0f30" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34665 } } peers { permanent_uuid: "dfd387fd9dc0445182b207e5f80ba735" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41901 } }
I20260929 03:03:00.146798 28266 ts_tablet_manager.cc:1428] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260929 03:03:00.148387 28247 heartbeater.cc:503] Master 127.8.208.126:39317 was elected leader, sending a full tablet report...
I20260929 03:03:00.149422 28267 ts_tablet_manager.cc:1428] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3: Time spent starting tablet: real 0.018s user 0.016s sys 0.003s
I20260929 03:03:00.149502 28117 heartbeater.cc:503] Master 127.8.208.126:39317 was elected leader, sending a full tablet report...
I20260929 03:03:00.150409 28265 ts_tablet_manager.cc:1428] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735: Time spent starting tablet: real 0.021s user 0.018s sys 0.001s
W20260929 03:03:00.262907 28248 tablet.cc:2369] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:03:00.317564 28118 tablet.cc:2369] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:03:00.357309 27987 tablet.cc:2369] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:03:00.473682 28273 raft_consensus.cc:482] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:03:00.473924 28273 raft_consensus.cc:504] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9afab83d1b9466bb94a219407479fd3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40007 } } peers { permanent_uuid: "18e2728c01a54512a3025cf287dd0f30" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34665 } } peers { permanent_uuid: "dfd387fd9dc0445182b207e5f80ba735" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41901 } }
I20260929 03:03:00.475538 28273 leader_election.cc:288] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9afab83d1b9466bb94a219407479fd3 (127.8.208.67:40007), dfd387fd9dc0445182b207e5f80ba735 (127.8.208.66:41901)
I20260929 03:03:00.483037 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.476000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:37683 (local address 127.8.208.67:40007)
0929 03:03:00.476204 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:00.476214 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:00.477047 (+ 833us) server_negotiation.cc:407] Connection header received
0929 03:03:00.477296 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:00.477309 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:00.477367 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:00.477437 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:00.478406 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.479249 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.481335 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.482057 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.482256 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.482330 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:00.482637 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 03:03:00.482765 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 03:03:00.483096 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.476154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:41175 (local address 127.8.208.66:41901)
0929 03:03:00.476415 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:00.476424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:00.477090 (+ 666us) server_negotiation.cc:407] Connection header received
0929 03:03:00.477300 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:00.477313 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:00.477399 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:00.477534 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:00.478278 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.479217 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.481260 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.482053 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.482242 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.482325 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:00.482693 (+ 368us) server_negotiation.cc:299] Negotiation successful
0929 03:03:00.482819 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260929 03:03:00.483142 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.476065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:41901 (local address 127.8.208.65:41175)
0929 03:03:00.476947 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:00.476961 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:00.477126 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:00.477569 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:00.477574 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:00.477604 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:00.478093 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:00.478103 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.479352 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.479358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:00.481047 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:00.481059 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.482306 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.482314 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:00.482509 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.482531 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 03:03:00.482647 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 03:03:00.482767 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":121,"thread_start_us":559,"threads_started":1}
I20260929 03:03:00.483269 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.475964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:40007 (local address 127.8.208.65:37683)
0929 03:03:00.476892 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:00.476915 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:00.477095 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:00.477496 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:00.477505 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:00.477534 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:00.478220 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:00.478237 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.479420 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.479424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:00.481092 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:00.481109 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.482256 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.482263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:00.482443 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.482496 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 03:03:00.482578 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 03:03:00.482721 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":24,"thread_start_us":452,"threads_started":1}
I20260929 03:03:00.484257 28072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9d682b80594e4fbddef59d9efff976" candidate_uuid: "18e2728c01a54512a3025cf287dd0f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfd387fd9dc0445182b207e5f80ba735" is_pre_election: true
I20260929 03:03:00.484352 28202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9d682b80594e4fbddef59d9efff976" candidate_uuid: "18e2728c01a54512a3025cf287dd0f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9afab83d1b9466bb94a219407479fd3" is_pre_election: true
I20260929 03:03:00.484826 28072 raft_consensus.cc:2454] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e2728c01a54512a3025cf287dd0f30 in term 0.
I20260929 03:03:00.484896 28202 raft_consensus.cc:2454] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18e2728c01a54512a3025cf287dd0f30 in term 0.
I20260929 03:03:00.485523 27877 leader_election.cc:302] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e2728c01a54512a3025cf287dd0f30, c9afab83d1b9466bb94a219407479fd3; no voters:
I20260929 03:03:00.485992 28273 raft_consensus.cc:2781] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:03:00.486142 28273 raft_consensus.cc:482] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:03:00.486253 28273 raft_consensus.cc:3037] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:00.488464 28273 raft_consensus.cc:504] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9afab83d1b9466bb94a219407479fd3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40007 } } peers { permanent_uuid: "18e2728c01a54512a3025cf287dd0f30" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34665 } } peers { permanent_uuid: "dfd387fd9dc0445182b207e5f80ba735" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41901 } }
I20260929 03:03:00.489080 28273 leader_election.cc:288] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [CANDIDATE]: Term 1 election: Requested vote from peers c9afab83d1b9466bb94a219407479fd3 (127.8.208.67:40007), dfd387fd9dc0445182b207e5f80ba735 (127.8.208.66:41901)
I20260929 03:03:00.489603 28202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9d682b80594e4fbddef59d9efff976" candidate_uuid: "18e2728c01a54512a3025cf287dd0f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9afab83d1b9466bb94a219407479fd3"
I20260929 03:03:00.489715 28072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9d682b80594e4fbddef59d9efff976" candidate_uuid: "18e2728c01a54512a3025cf287dd0f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfd387fd9dc0445182b207e5f80ba735"
I20260929 03:03:00.489821 28202 raft_consensus.cc:3037] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:00.489897 28072 raft_consensus.cc:3037] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:00.493064 28072 raft_consensus.cc:2454] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e2728c01a54512a3025cf287dd0f30 in term 1.
I20260929 03:03:00.493058 28202 raft_consensus.cc:2454] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18e2728c01a54512a3025cf287dd0f30 in term 1.
I20260929 03:03:00.493470 27877 leader_election.cc:302] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18e2728c01a54512a3025cf287dd0f30, c9afab83d1b9466bb94a219407479fd3; no voters:
I20260929 03:03:00.493778 28273 raft_consensus.cc:2781] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:00.494221 28273 raft_consensus.cc:686] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 1 LEADER]: Becoming Leader. State: Replica: 18e2728c01a54512a3025cf287dd0f30, State: Running, Role: LEADER
I20260929 03:03:00.494704 28273 consensus_queue.cc:237] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9afab83d1b9466bb94a219407479fd3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40007 } } peers { permanent_uuid: "18e2728c01a54512a3025cf287dd0f30" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34665 } } peers { permanent_uuid: "dfd387fd9dc0445182b207e5f80ba735" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41901 } }
I20260929 03:03:00.499186 27799 catalog_manager.cc:5495] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18e2728c01a54512a3025cf287dd0f30 (127.8.208.65). New cstate: current_term: 1 leader_uuid: "18e2728c01a54512a3025cf287dd0f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9afab83d1b9466bb94a219407479fd3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18e2728c01a54512a3025cf287dd0f30" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dfd387fd9dc0445182b207e5f80ba735" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41901 } health_report { overall_health: UNKNOWN } } }
I20260929 03:03:00.569857 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.563571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50716 (local address 127.8.208.65:34665)
0929 03:03:00.563843 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:00.563852 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:00.564415 (+ 563us) server_negotiation.cc:407] Connection header received
0929 03:03:00.564656 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:00.564661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:00.564751 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:00.564900 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:00.565434 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.566135 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.567562 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.567975 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.568117 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.568726 (+ 609us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:00.569260 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:00.569436 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:00.569570 (+ 134us) server_negotiation.cc:299] Negotiation successful
0929 03:03:00.569615 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 03:03:00.578755 28202 raft_consensus.cc:1261] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3 [term 1 FOLLOWER]: Refusing update from remote peer 18e2728c01a54512a3025cf287dd0f30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:03:00.578756 28072 raft_consensus.cc:1261] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735 [term 1 FOLLOWER]: Refusing update from remote peer 18e2728c01a54512a3025cf287dd0f30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:03:00.579756 28276 consensus_queue.cc:1035] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfd387fd9dc0445182b207e5f80ba735" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 41901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:03:00.580084 28273 consensus_queue.cc:1035] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9afab83d1b9466bb94a219407479fd3" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:03:00.590428 28280 mvcc.cc:205] Tried to move back new op lower bound from 7334506416440516608 to 7334506416109969408. Current Snapshot: MvccSnapshot[applied={T|T < 7334506416440516608}]
I20260929 03:03:00.597396 28282 mvcc.cc:205] Tried to move back new op lower bound from 7334506416440516608 to 7334506416109969408. Current Snapshot: MvccSnapshot[applied={T|T < 7334506416440516608}]
I20260929 03:03:00.607496 27800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:03:00.607787 27800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:03:00.611306 27800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c452b02f370b45b290f3d326502d3d21: Sending DeleteTablet for 3 replicas of tablet fe9d682b80594e4fbddef59d9efff976
I20260929 03:03:00.612429 27921 tablet_service.cc:1514] Processing DeleteTablet for tablet fe9d682b80594e4fbddef59d9efff976 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:03:00 UTC) from {username='slave'} at 127.0.0.1:50708
I20260929 03:03:00.612439 28182 tablet_service.cc:1514] Processing DeleteTablet for tablet fe9d682b80594e4fbddef59d9efff976 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:03:00 UTC) from {username='slave'} at 127.0.0.1:56106
I20260929 03:03:00.612905 28052 tablet_service.cc:1514] Processing DeleteTablet for tablet fe9d682b80594e4fbddef59d9efff976 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:03:00 UTC) from {username='slave'} at 127.0.0.1:55418
I20260929 03:03:00.615115 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 27859
I20260929 03:03:00.644970 28292 tablet_replica.cc:330] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735: stopping tablet replica
I20260929 03:03:00.650161 28292 raft_consensus.cc:2229] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:03:00.650954 28292 raft_consensus.cc:2258] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:03:00.654548 28292 ts_tablet_manager.cc:1905] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:03:00.655354 28290 tablet_replica.cc:330] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30: stopping tablet replica
I20260929 03:03:00.655930 28290 raft_consensus.cc:2229] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:03:00.656713 28290 raft_consensus.cc:2258] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:03:00.658759 28290 ts_tablet_manager.cc:1905] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:03:00.666790 28290 ts_tablet_manager.cc:1918] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:03:00.667038 28290 log.cc:1196] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/fe9d682b80594e4fbddef59d9efff976
I20260929 03:03:00.667676 28290 ts_tablet_manager.cc:1939] T fe9d682b80594e4fbddef59d9efff976 P 18e2728c01a54512a3025cf287dd0f30: Deleting consensus metadata
I20260929 03:03:00.669655 27784 catalog_manager.cc:4841] TS 18e2728c01a54512a3025cf287dd0f30 (127.8.208.65:34665): tablet fe9d682b80594e4fbddef59d9efff976 (table test-table [id=c4688adaeadf4895bb190fc83655d52c]) successfully deleted
I20260929 03:03:00.670006 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.620028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50718 (local address 127.8.208.65:34665)
0929 03:03:00.620270 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:00.620279 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:00.643085 (+ 22806us) server_negotiation.cc:407] Connection header received
0929 03:03:00.643694 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:00.643703 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:00.643837 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:00.643933 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:00.645711 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.647336 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.657218 (+ 9882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.658192 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.658309 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.660260 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:00.660274 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:00.660300 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:00.661429 (+ 1129us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:00.661507 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:00.661537 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:00.661657 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:00.669348 (+ 7691us) server_negotiation.cc:299] Negotiation successful
0929 03:03:00.669511 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 03:03:00.671329 28291 tablet_replica.cc:330] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3: stopping tablet replica
I20260929 03:03:00.671483 27981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:00.671916 28291 raft_consensus.cc:2229] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:03:00.672873 28291 raft_consensus.cc:2258] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:03:00.673651 28292 ts_tablet_manager.cc:1918] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:03:00.673887 28292 log.cc:1196] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/fe9d682b80594e4fbddef59d9efff976
I20260929 03:03:00.674492 28292 ts_tablet_manager.cc:1939] T fe9d682b80594e4fbddef59d9efff976 P dfd387fd9dc0445182b207e5f80ba735: Deleting consensus metadata
I20260929 03:03:00.675639 28291 ts_tablet_manager.cc:1905] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:03:00.676494 27786 catalog_manager.cc:4841] TS dfd387fd9dc0445182b207e5f80ba735 (127.8.208.66:41901): tablet fe9d682b80594e4fbddef59d9efff976 (table test-table [id=c4688adaeadf4895bb190fc83655d52c]) successfully deleted
I20260929 03:03:00.680630 28291 ts_tablet_manager.cc:1918] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:03:00.680811 28291 log.cc:1196] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/fe9d682b80594e4fbddef59d9efff976
I20260929 03:03:00.681264 28291 ts_tablet_manager.cc:1939] T fe9d682b80594e4fbddef59d9efff976 P c9afab83d1b9466bb94a219407479fd3: Deleting consensus metadata
I20260929 03:03:00.682672 27787 catalog_manager.cc:4841] TS c9afab83d1b9466bb94a219407479fd3 (127.8.208.67:40007): tablet fe9d682b80594e4fbddef59d9efff976 (table test-table [id=c4688adaeadf4895bb190fc83655d52c]) successfully deleted
I20260929 03:03:00.771142 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 27859
I20260929 03:03:00.790386 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 27990
I20260929 03:03:00.797417 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.791200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55426 (local address 127.8.208.66:41901)
0929 03:03:00.791488 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:00.791496 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:00.791520 (+ 24us) server_negotiation.cc:407] Connection header received
0929 03:03:00.791749 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:00.791753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:00.791832 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:00.791937 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:00.792322 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.793070 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.794397 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.794818 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.794959 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.796000 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:00.796009 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:00.796023 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:00.796597 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:00.796657 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:00.796675 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:00.796796 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:00.796927 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 03:03:00.797044 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260929 03:03:00.798130 28112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:00.887434 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 27990
I20260929 03:03:00.905380 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 28120
I20260929 03:03:00.912400 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:00.906297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.8.208.67:40007)
0929 03:03:00.906471 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:00.906476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:00.906495 (+ 19us) server_negotiation.cc:407] Connection header received
0929 03:03:00.906601 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:00.906605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:00.906683 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:00.906771 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:00.907174 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.908010 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.909400 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:00.909824 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:00.909923 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:00.911033 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:00.911042 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:00.911057 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:00.911633 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:00.911680 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:00.911701 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:00.911787 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:00.911930 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 03:03:00.912019 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 03:03:00.913062 28242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:01.000360 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 28120
I20260929 03:03:01.016877 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 27769
I20260929 03:03:01.018013 27830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:01.126847 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 27769
2026-09-29T03:03:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3083 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:01 dist-test-slave-m1rv krb5kdc[28300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:01 dist-test-slave-m1rv krb5kdc[28300](info): set up 2 sockets
Sep 29 03:03:01 dist-test-slave-m1rv krb5kdc[28300](info): commencing operation
krb5kdc: starting...
W20260929 03:03:02.241631 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:02 dist-test-slave-m1rv krb5kdc[28300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:03:02.486472 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:35455
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:39991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:35455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:02.811087 28313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:02.811740 28313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:02.811901 28313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:02.824052 28313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:02.824201 28313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:02.824294 28313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:02.824388 28313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:02.824471 28313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:02.841804 28313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:39991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:35455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:35455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:02.843922 28313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:02.845970 28313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:02.860996 28319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:02.861243 28321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:02.861129 28318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:02.861820 28313 server_base.cc:1034] running on GCE node
I20260929 03:03:02.862614 28313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:02.864028 28313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:02.865278 28313 hybrid_clock.cc:648] HybridClock initialized: now 1790650982865236 us; error 57 us; skew 500 ppm
I20260929 03:03:02.865625 28313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:02 dist-test-slave-m1rv krb5kdc[28300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650982, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:02.868724 28313 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:03:02.870220 28313 webserver.cc:460] Webserver started at http://127.8.208.126:39133/ using document root <none> and password file <none>
I20260929 03:03:02.870988 28313 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:02.871140 28313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:02.871447 28313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:02.874431 28313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "207f14a70db34b238390c8d90a2d1692"
format_stamp: "Formatted at 2026-09-29 03:03:02 on dist-test-slave-m1rv"
server_key: "c8cf7f134f1d2db2db325120e9ca7445"
server_key_iv: "d2d1c8041f62df09454f61c252afbe78"
server_key_version: "encryptionkey@0"
I20260929 03:03:02.875164 28313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "207f14a70db34b238390c8d90a2d1692"
format_stamp: "Formatted at 2026-09-29 03:03:02 on dist-test-slave-m1rv"
server_key: "c8cf7f134f1d2db2db325120e9ca7445"
server_key_iv: "d2d1c8041f62df09454f61c252afbe78"
server_key_version: "encryptionkey@0"
I20260929 03:03:02.880091 28313 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 03:03:02.883378 28328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:02.884558 28313 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 03:03:02.884739 28313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "207f14a70db34b238390c8d90a2d1692"
format_stamp: "Formatted at 2026-09-29 03:03:02 on dist-test-slave-m1rv"
server_key: "c8cf7f134f1d2db2db325120e9ca7445"
server_key_iv: "d2d1c8041f62df09454f61c252afbe78"
server_key_version: "encryptionkey@0"
I20260929 03:03:02.884928 28313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:02.905097 28313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:02.908917 28313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:02.909221 28313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:02.927471 28313 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:35455
I20260929 03:03:02.927512 28379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:35455 every 8 connection(s)
I20260929 03:03:02.928822 28313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:02.929498 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 28313
I20260929 03:03:02.929678 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:02.930308 9025 external_mini_cluster.cc:1428] Setting key e2e5553965370798f1187b0ac3e05e6f
I20260929 03:03:02.933334 28380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 03:03:02 dist-test-slave-m1rv krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:03:02.943717 28380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692: Bootstrap starting.
I20260929 03:03:02.947649 28380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:02.948972 28380 log.cc:824] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:02.952337 28380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692: No bootstrap required, opened a new log
I20260929 03:03:02.953398 28383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:02.932188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53742 (local address 127.8.208.126:35455)
0929 03:03:02.932929 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:02.932954 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:02.932993 (+ 39us) server_negotiation.cc:407] Connection header received
0929 03:03:02.934747 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:02.934767 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:02.935120 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:02.935498 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:02.937049 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:02.938442 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:02.939741 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:02.940409 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:02.940632 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:02.944670 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:02.944679 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:02.944696 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:02.944746 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:02.948683 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:02.949541 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:02.949553 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:02.949565 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:02.949768 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:02.950273 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:02.950278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:02.950280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:02.950896 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:02.951200 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:02.951561 (+ 361us) server_negotiation.cc:299] Negotiation successful
0929 03:03:02.951989 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":548,"thread_start_us":343,"threads_started":1}
I20260929 03:03:02.960596 28380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207f14a70db34b238390c8d90a2d1692" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35455 } }
I20260929 03:03:02.960937 28380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:02.961009 28380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 207f14a70db34b238390c8d90a2d1692, State: Initialized, Role: FOLLOWER
I20260929 03:03:02.961582 28380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207f14a70db34b238390c8d90a2d1692" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35455 } }
I20260929 03:03:02.961807 28380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:02.961937 28380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:02.962152 28380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:02.964395 28380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207f14a70db34b238390c8d90a2d1692" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35455 } }
I20260929 03:03:02.964820 28380 leader_election.cc:302] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 207f14a70db34b238390c8d90a2d1692; no voters:
I20260929 03:03:02.965400 28380 leader_election.cc:288] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:02.965549 28385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:02.966224 28385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [term 1 LEADER]: Becoming Leader. State: Replica: 207f14a70db34b238390c8d90a2d1692, State: Running, Role: LEADER
I20260929 03:03:02.966759 28385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207f14a70db34b238390c8d90a2d1692" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35455 } }
I20260929 03:03:02.967442 28380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:02.970031 28386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "207f14a70db34b238390c8d90a2d1692" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207f14a70db34b238390c8d90a2d1692" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35455 } } }
I20260929 03:03:02.970485 28386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:02.971226 28387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 207f14a70db34b238390c8d90a2d1692. Latest consensus state: current_term: 1 leader_uuid: "207f14a70db34b238390c8d90a2d1692" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207f14a70db34b238390c8d90a2d1692" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35455 } } }
I20260929 03:03:02.972715 28387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:02.972762 28393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:02.977921 28393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:02.988229 28393 catalog_manager.cc:1347] Generated new cluster ID: 1ce1d115f54b4216b4ad55c29ddc72a8
I20260929 03:03:02.988363 28393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:02.997852 28393 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:02.999621 28393 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:03.009234 28393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692: Generated new TSK 0
I20260929 03:03:03.010058 28393 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.208.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
Entry for principal kudu/127.8.208.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab.
I20260929 03:03:03.084609 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35455
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:39991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:03.416788 28406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:03.417081 28406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:03.417263 28406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:03.424757 28406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:03.424875 28406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:03.425148 28406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:03:03.435051 28406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:39991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.65.keytab
--principal=kudu/127.8.208.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:03.436388 28406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:03.437744 28406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:03.450831 28412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:03.450871 28414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:03.450831 28411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:03.452056 28406 server_base.cc:1034] running on GCE node
I20260929 03:03:03.452673 28406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:03.453761 28406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:03.455040 28406 hybrid_clock.cc:648] HybridClock initialized: now 1790650983454983 us; error 57 us; skew 500 ppm
I20260929 03:03:03.455467 28406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:03 dist-test-slave-m1rv krb5kdc[28300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650983, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:03.459151 28406 init.cc:377] Logged in from keytab as kudu/127.8.208.65@KRBTEST.COM (short username kudu)
I20260929 03:03:03.460526 28406 webserver.cc:460] Webserver started at http://127.8.208.65:42085/ using document root <none> and password file <none>
I20260929 03:03:03.461163 28406 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:03.461331 28406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:03.461594 28406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:03.464448 28406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "9fe85ae9d98c4fa29991789e257670af"
format_stamp: "Formatted at 2026-09-29 03:03:03 on dist-test-slave-m1rv"
server_key: "87b1296fe2ecc9457e2c7f61d33ced51"
server_key_iv: "adf3627ce796a4fed0c7d5fa2292b588"
server_key_version: "encryptionkey@0"
I20260929 03:03:03.465217 28406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "9fe85ae9d98c4fa29991789e257670af"
format_stamp: "Formatted at 2026-09-29 03:03:03 on dist-test-slave-m1rv"
server_key: "87b1296fe2ecc9457e2c7f61d33ced51"
server_key_iv: "adf3627ce796a4fed0c7d5fa2292b588"
server_key_version: "encryptionkey@0"
I20260929 03:03:03.470352 28406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 03:03:03.474043 28421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:03.475469 28406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:03:03.475744 28406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "9fe85ae9d98c4fa29991789e257670af"
format_stamp: "Formatted at 2026-09-29 03:03:03 on dist-test-slave-m1rv"
server_key: "87b1296fe2ecc9457e2c7f61d33ced51"
server_key_iv: "adf3627ce796a4fed0c7d5fa2292b588"
server_key_version: "encryptionkey@0"
I20260929 03:03:03.475939 28406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:03.500046 28406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:03.504030 28406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:03.504318 28406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:03.505297 28406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:03:03.506717 28406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:03:03.506821 28406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:03.506954 28406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:03:03.507017 28406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:03.538789 28406 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:34671
I20260929 03:03:03.538826 28533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:34671 every 8 connection(s)
I20260929 03:03:03.540077 28406 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:03:03.541522 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 28406
I20260929 03:03:03.541711 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 03:03:03.542155 9025 external_mini_cluster.cc:1428] Setting key ad9b0345c8c6e36f5406554bf916c77b
Sep 29 03:03:03 dist-test-slave-m1rv krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650983, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.65@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:03:03.563916 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:03.543030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:50881 (local address 127.8.208.126:35455)
0929 03:03:03.544326 (+ 1296us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:03.544336 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:03.545128 (+ 792us) server_negotiation.cc:407] Connection header received
0929 03:03:03.548052 (+ 2924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:03.548056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:03.548121 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:03.548216 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:03.550305 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:03.551001 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:03.553509 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:03.554010 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:03.554162 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:03.558075 (+ 3913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:03.558080 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:03.558083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:03.558116 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:03.560735 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:03.561534 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:03.561540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:03.561544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:03.561668 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:03.562108 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:03.562113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:03.562117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:03.562379 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:03.562543 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:03.563248 (+ 705us) server_negotiation.cc:299] Negotiation successful
0929 03:03:03.563488 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1132,"thread_start_us":986,"threads_started":1}
I20260929 03:03:03.564828 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:03.543618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35455 (local address 127.8.208.65:50881)
0929 03:03:03.544941 (+ 1323us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:03.544999 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:03.547763 (+ 2764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:03.548478 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:03.548491 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:03.549298 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:03.550125 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:03.550153 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:03.551165 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:03.551173 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:03.553343 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:03.553362 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:03.554237 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:03.554242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:03.554433 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:03.555306 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:03.555333 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:03.557810 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:03.560876 (+ 3066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:03.560885 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:03.560902 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:03.561406 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:03.561778 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:03.561784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:03.561787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:03.561980 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:03.562525 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:03.562533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:03.562869 (+ 336us) client_negotiation.cc:769] Sending connection context
0929 03:03:03.563143 (+ 274us) client_negotiation.cc:240] Negotiation successful
0929 03:03:03.563673 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1012,"thread_start_us":784,"threads_started":1}
I20260929 03:03:03.566561 28534 heartbeater.cc:347] Connected to a master server at 127.8.208.126:35455
I20260929 03:03:03.566913 28534 heartbeater.cc:464] Registering TS with master...
I20260929 03:03:03.567865 28534 heartbeater.cc:511] Master 127.8.208.126:35455 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.208.66@KRBTEST.COM; defaulting to no policy
I20260929 03:03:03.570133 28345 ts_manager.cc:194] Registered new tserver with Master: 9fe85ae9d98c4fa29991789e257670af (127.8.208.65:34671)
I20260929 03:03:03.571723 28345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.65@KRBTEST.COM'} at 127.8.208.65:50881
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
Entry for principal kudu/127.8.208.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab.
I20260929 03:03:03.600965 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35455
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:39991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:03.928658 28540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:03.929032 28540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:03.929203 28540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:03.940776 28540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:03.940918 28540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:03.941090 28540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:03:03.954317 28540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:39991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.66.keytab
--principal=kudu/127.8.208.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:03.955545 28540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:03.956795 28540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:03.970775 28546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:03.970858 28548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:03.970810 28545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:03.971477 28540 server_base.cc:1034] running on GCE node
I20260929 03:03:03.972092 28540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:03.973115 28540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:03.974364 28540 hybrid_clock.cc:648] HybridClock initialized: now 1790650983974310 us; error 48 us; skew 500 ppm
I20260929 03:03:03.974694 28540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:03 dist-test-slave-m1rv krb5kdc[28300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650983, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:03.977984 28540 init.cc:377] Logged in from keytab as kudu/127.8.208.66@KRBTEST.COM (short username kudu)
I20260929 03:03:03.979593 28540 webserver.cc:460] Webserver started at http://127.8.208.66:44727/ using document root <none> and password file <none>
I20260929 03:03:03.980384 28540 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:03.980553 28540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:03.980867 28540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:03.983989 28540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "9eccb1f3a7e74a54913a45a9d2d8101e"
format_stamp: "Formatted at 2026-09-29 03:03:03 on dist-test-slave-m1rv"
server_key: "2a1e29909b983e269f28187e6e637962"
server_key_iv: "5ebc9daddefd44983b0e36e6b97e9130"
server_key_version: "encryptionkey@0"
I20260929 03:03:03.985170 28540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "9eccb1f3a7e74a54913a45a9d2d8101e"
format_stamp: "Formatted at 2026-09-29 03:03:03 on dist-test-slave-m1rv"
server_key: "2a1e29909b983e269f28187e6e637962"
server_key_iv: "5ebc9daddefd44983b0e36e6b97e9130"
server_key_version: "encryptionkey@0"
I20260929 03:03:03.990334 28540 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 03:03:03.993628 28555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:03.994853 28540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260929 03:03:03.995090 28540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "9eccb1f3a7e74a54913a45a9d2d8101e"
format_stamp: "Formatted at 2026-09-29 03:03:03 on dist-test-slave-m1rv"
server_key: "2a1e29909b983e269f28187e6e637962"
server_key_iv: "5ebc9daddefd44983b0e36e6b97e9130"
server_key_version: "encryptionkey@0"
I20260929 03:03:03.995333 28540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:04.012631 28540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:04.016973 28540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:04.017323 28540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:04.018306 28540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:03:04.019629 28540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:03:04.019737 28540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:04.019891 28540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:03:04.019973 28540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:04.052614 28540 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:34693
I20260929 03:03:04.052655 28667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:34693 every 8 connection(s)
I20260929 03:03:04.053985 28540 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:03:04.054457 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 28540
I20260929 03:03:04.054642 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 03:03:04.055294 9025 external_mini_cluster.cc:1428] Setting key 003403bab1b2140cb502325444495348
Sep 29 03:03:04 dist-test-slave-m1rv krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650983, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.66@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:03:04.076207 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.057094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:41201 (local address 127.8.208.126:35455)
0929 03:03:04.057342 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:04.057348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:04.058554 (+ 1206us) server_negotiation.cc:407] Connection header received
0929 03:03:04.060363 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:04.060368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:04.060450 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:04.060571 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:04.062133 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.062843 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.064366 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.064786 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.064922 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.069220 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:04.069230 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:04.069238 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:04.069284 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:04.072919 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:04.073693 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:04.073701 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:04.073707 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:04.073863 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:04.074263 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:04.074270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:04.074277 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:04.074683 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:04.074845 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:04.075490 (+ 645us) server_negotiation.cc:299] Negotiation successful
0929 03:03:04.075755 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260929 03:03:04.076869 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.057525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35455 (local address 127.8.208.66:41201)
0929 03:03:04.058386 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:04.058424 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:04.060078 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:04.060803 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:04.060812 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:04.061397 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:04.061968 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.061986 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.063000 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.063012 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.064213 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.064224 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.065021 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.065028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.065319 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.066108 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:04.066134 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:04.068958 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:04.073065 (+ 4107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:04.073071 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:04.073083 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:04.073564 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:04.073966 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:04.073969 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:04.073971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:04.074130 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:04.074797 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:04.074803 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:04.075108 (+ 305us) client_negotiation.cc:769] Sending connection context
0929 03:03:04.075334 (+ 226us) client_negotiation.cc:240] Negotiation successful
0929 03:03:04.075711 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":369,"threads_started":1}
I20260929 03:03:04.078346 28668 heartbeater.cc:347] Connected to a master server at 127.8.208.126:35455
I20260929 03:03:04.078677 28668 heartbeater.cc:464] Registering TS with master...
I20260929 03:03:04.079579 28668 heartbeater.cc:511] Master 127.8.208.126:35455 requested a full tablet report, sending...
I20260929 03:03:04.081319 28345 ts_manager.cc:194] Registered new tserver with Master: 9eccb1f3a7e74a54913a45a9d2d8101e (127.8.208.66:34693)
I20260929 03:03:04.082544 28345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.66@KRBTEST.COM'} at 127.8.208.66:41201
WARNING: no policy specified for kudu/127.8.208.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
Entry for principal kudu/127.8.208.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab.
I20260929 03:03:04.116127 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35455
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:39991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:04.437783 28673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:04.438086 28673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:04.438279 28673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:04.445315 28673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:04.445467 28673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:04.445665 28673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:03:04.455212 28673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:39991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.67.keytab
--principal=kudu/127.8.208.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:35455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:04.456506 28673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:04.457834 28673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:04.470990 28679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:04.470992 28678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:04.471122 28681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:04.471906 28673 server_base.cc:1034] running on GCE node
I20260929 03:03:04.472455 28673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:04.473490 28673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:04.474687 28673 hybrid_clock.cc:648] HybridClock initialized: now 1790650984474648 us; error 33 us; skew 500 ppm
I20260929 03:03:04.474970 28673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:04 dist-test-slave-m1rv krb5kdc[28300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650984, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:04.477989 28673 init.cc:377] Logged in from keytab as kudu/127.8.208.67@KRBTEST.COM (short username kudu)
I20260929 03:03:04.479393 28673 webserver.cc:460] Webserver started at http://127.8.208.67:41705/ using document root <none> and password file <none>
I20260929 03:03:04.480140 28673 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:04.480274 28673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:04.480592 28673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:04.483533 28673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1"
format_stamp: "Formatted at 2026-09-29 03:03:04 on dist-test-slave-m1rv"
server_key: "9ceec612f55e782c66f227ffff355107"
server_key_iv: "2da53666d3c46776502d409fb55c83f0"
server_key_version: "encryptionkey@0"
I20260929 03:03:04.484383 28673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1"
format_stamp: "Formatted at 2026-09-29 03:03:04 on dist-test-slave-m1rv"
server_key: "9ceec612f55e782c66f227ffff355107"
server_key_iv: "2da53666d3c46776502d409fb55c83f0"
server_key_version: "encryptionkey@0"
I20260929 03:03:04.489847 28673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260929 03:03:04.493388 28688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:04.494710 28673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:03:04.494868 28673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1"
format_stamp: "Formatted at 2026-09-29 03:03:04 on dist-test-slave-m1rv"
server_key: "9ceec612f55e782c66f227ffff355107"
server_key_iv: "2da53666d3c46776502d409fb55c83f0"
server_key_version: "encryptionkey@0"
I20260929 03:03:04.495071 28673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:04.507792 28673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:04.512181 28673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:04.512481 28673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:04.513533 28673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:03:04.514879 28673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:03:04.514950 28673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:04.515096 28673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:03:04.515146 28673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:04.548108 28673 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:39477
I20260929 03:03:04.548154 28800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:39477 every 8 connection(s)
I20260929 03:03:04.549433 28673 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:03:04.559073 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 28673
I20260929 03:03:04.559269 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 03:03:04.559811 9025 external_mini_cluster.cc:1428] Setting key b6c4ec38df7452064cd80dd5d51f7b2d
Sep 29 03:03:04 dist-test-slave-m1rv krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650984, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.67@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:03:04.569152 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.552077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:55291 (local address 127.8.208.126:35455)
0929 03:03:04.552258 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:04.552262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:04.553341 (+ 1079us) server_negotiation.cc:407] Connection header received
0929 03:03:04.555190 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:04.555196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:04.555315 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:04.555430 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:04.556886 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.557701 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.559567 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.560030 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.560133 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.563715 (+ 3582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:04.563721 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:04.563724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:04.563758 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:04.566333 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:04.567008 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:04.567012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:04.567014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:04.567142 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:04.567511 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:04.567514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:04.567516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:04.567771 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:04.567986 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:04.568650 (+ 664us) server_negotiation.cc:299] Negotiation successful
0929 03:03:04.568810 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 03:03:04.570098 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.552459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:35455 (local address 127.8.208.67:55291)
0929 03:03:04.553132 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:04.553211 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:04.554926 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:04.555669 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:04.555677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:04.556158 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:04.556715 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.556731 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.557855 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.557860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.559394 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.559409 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.560251 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.560259 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.560532 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.561339 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:04.561368 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:04.563524 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:04.566465 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:04.566471 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:04.566484 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:04.566905 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:04.567238 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:04.567241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:04.567243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:04.567416 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:04.567958 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:04.567964 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:04.568301 (+ 337us) client_negotiation.cc:769] Sending connection context
0929 03:03:04.568543 (+ 242us) client_negotiation.cc:240] Negotiation successful
0929 03:03:04.568976 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":262,"threads_started":1}
I20260929 03:03:04.571521 28801 heartbeater.cc:347] Connected to a master server at 127.8.208.126:35455
I20260929 03:03:04.571828 28801 heartbeater.cc:464] Registering TS with master...
I20260929 03:03:04.572492 28801 heartbeater.cc:511] Master 127.8.208.126:35455 requested a full tablet report, sending...
I20260929 03:03:04.574062 28344 ts_manager.cc:194] Registered new tserver with Master: 5f30f1c475a648a4bb3e72ac04a8ddd1 (127.8.208.67:39477)
I20260929 03:03:04.575071 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:03:04.575237 28344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.208.67@KRBTEST.COM'} at 127.8.208.67:55291
I20260929 03:03:04.575310 28534 heartbeater.cc:503] Master 127.8.208.126:35455 was elected leader, sending a full tablet report...
I20260929 03:03:04.592031 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.579446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53754 (local address 127.8.208.126:35455)
0929 03:03:04.579613 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:04.579621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:04.580086 (+ 465us) server_negotiation.cc:407] Connection header received
0929 03:03:04.580321 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:04.580326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:04.580381 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:04.580481 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:04.581966 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.582684 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.583929 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.584332 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.584472 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.586887 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:04.586893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:04.586897 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:04.586936 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:04.589510 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:04.590095 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:04.590101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:04.590105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:04.590229 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:04.590578 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:04.590585 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:04.590591 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:04.590913 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:04.591184 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:04.591474 (+ 290us) server_negotiation.cc:299] Negotiation successful
0929 03:03:04.591694 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 03:03:04.595501 28344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:03:04.602519 28344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:03:04.639945 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.614938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:34693 (local address 127.0.0.1:53124)
0929 03:03:04.615950 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:04.616006 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:04.616212 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:04.628195 (+ 11983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:04.628207 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:04.628262 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:04.628835 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.628851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.631739 (+ 2888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.631747 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.634981 (+ 3234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.635002 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.638856 (+ 3854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.638863 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.639122 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.639227 (+ 105us) client_negotiation.cc:769] Sending connection context
0929 03:03:04.639364 (+ 137us) client_negotiation.cc:240] Negotiation successful
0929 03:03:04.639568 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":1554,"threads_started":1}
I20260929 03:03:04.640190 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.615425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53124 (local address 127.8.208.66:34693)
0929 03:03:04.627080 (+ 11655us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:04.627095 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:04.627132 (+ 37us) server_negotiation.cc:407] Connection header received
0929 03:03:04.627258 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:04.627270 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:04.627662 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:04.629102 (+ 1440us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:04.629391 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.631530 (+ 2139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.635255 (+ 3725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.637373 (+ 2118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.639375 (+ 2002us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.639613 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:04.639694 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 03:03:04.639860 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11429,"thread_start_us":12418,"threads_started":1}
I20260929 03:03:04.644219 28603 tablet_service.cc:1467] Processing CreateTablet for tablet bdc2d4cdab3f44738934c4546e232208 (DEFAULT_TABLE table=test-table [id=21d97f71be5a41f1bc8c8c05f0c6363f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:03:04.645464 28817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.615309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:34671 (local address 127.0.0.1:57544)
0929 03:03:04.625549 (+ 10240us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:04.625576 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:04.625740 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:04.638567 (+ 12827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:04.638572 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:04.638598 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:04.639112 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.639123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.641444 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.641449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.643253 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.643267 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.644532 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.644538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.644719 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.644756 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 03:03:04.644871 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 03:03:04.645002 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10032,"mutex_wait_us":1,"thread_start_us":22444,"threads_started":1}
I20260929 03:03:04.645509 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.615419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57544 (local address 127.8.208.65:34671)
0929 03:03:04.638073 (+ 22654us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:04.638079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:04.638104 (+ 25us) server_negotiation.cc:407] Connection header received
0929 03:03:04.638180 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:04.638186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:04.638375 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:04.638534 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:04.639316 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.641280 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.643405 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.644328 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.644481 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.644726 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:04.644994 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 03:03:04.645200 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22476,"thread_start_us":22314,"threads_started":1}
I20260929 03:03:04.645792 28603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdc2d4cdab3f44738934c4546e232208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:04.646134 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.615931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:39477 (local address 127.0.0.1:36722)
0929 03:03:04.638024 (+ 22093us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:04.638046 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:04.638194 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:04.638799 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:04.638803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:04.638838 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:04.639341 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.639355 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.641033 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.641041 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.643307 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.643322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.645332 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.645341 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.645538 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.645572 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 03:03:04.645662 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 03:03:04.645789 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21960,"mutex_wait_us":69,"thread_start_us":9223,"threads_started":1}
I20260929 03:03:04.646353 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.616648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36722 (local address 127.8.208.67:39477)
0929 03:03:04.634119 (+ 17471us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:04.634132 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:04.638160 (+ 4028us) server_negotiation.cc:407] Connection header received
0929 03:03:04.638339 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:04.638347 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:04.638567 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:04.638778 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:04.639535 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.640880 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.643500 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.644822 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.644942 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.645157 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:04.645854 (+ 697us) server_negotiation.cc:299] Negotiation successful
0929 03:03:04.646014 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17232,"thread_start_us":17480,"threads_started":1}
I20260929 03:03:04.649581 28736 tablet_service.cc:1467] Processing CreateTablet for tablet bdc2d4cdab3f44738934c4546e232208 (DEFAULT_TABLE table=test-table [id=21d97f71be5a41f1bc8c8c05f0c6363f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:03:04.649482 28469 tablet_service.cc:1467] Processing CreateTablet for tablet bdc2d4cdab3f44738934c4546e232208 (DEFAULT_TABLE table=test-table [id=21d97f71be5a41f1bc8c8c05f0c6363f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:03:04.650717 28736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdc2d4cdab3f44738934c4546e232208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:04.650959 28469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdc2d4cdab3f44738934c4546e232208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:04.660490 28819 tablet_bootstrap.cc:492] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e: Bootstrap starting.
I20260929 03:03:04.662377 28820 tablet_bootstrap.cc:492] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1: Bootstrap starting.
I20260929 03:03:04.665026 28819 tablet_bootstrap.cc:654] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:04.665853 28821 tablet_bootstrap.cc:492] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af: Bootstrap starting.
I20260929 03:03:04.665922 28820 tablet_bootstrap.cc:654] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:04.666575 28819 log.cc:824] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:04.667013 28820 log.cc:824] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:04.669596 28821 tablet_bootstrap.cc:654] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:04.670222 28819 tablet_bootstrap.cc:492] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e: No bootstrap required, opened a new log
I20260929 03:03:04.670332 28820 tablet_bootstrap.cc:492] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1: No bootstrap required, opened a new log
I20260929 03:03:04.670539 28819 ts_tablet_manager.cc:1397] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260929 03:03:04.670616 28820 ts_tablet_manager.cc:1397] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 03:03:04.670771 28821 log.cc:824] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:04.673695 28821 tablet_bootstrap.cc:492] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af: No bootstrap required, opened a new log
I20260929 03:03:04.673942 28821 ts_tablet_manager.cc:1397] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260929 03:03:04.678751 28819 raft_consensus.cc:348] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 34693 } } peers { permanent_uuid: "9fe85ae9d98c4fa29991789e257670af" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34671 } } peers { permanent_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39477 } }
I20260929 03:03:04.679163 28819 raft_consensus.cc:374] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:04.679335 28819 raft_consensus.cc:729] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eccb1f3a7e74a54913a45a9d2d8101e, State: Initialized, Role: FOLLOWER
I20260929 03:03:04.679994 28819 consensus_queue.cc:260] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 34693 } } peers { permanent_uuid: "9fe85ae9d98c4fa29991789e257670af" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34671 } } peers { permanent_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39477 } }
I20260929 03:03:04.680915 28820 raft_consensus.cc:348] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 34693 } } peers { permanent_uuid: "9fe85ae9d98c4fa29991789e257670af" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34671 } } peers { permanent_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39477 } }
I20260929 03:03:04.681329 28820 raft_consensus.cc:374] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:04.681422 28820 raft_consensus.cc:729] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f30f1c475a648a4bb3e72ac04a8ddd1, State: Initialized, Role: FOLLOWER
I20260929 03:03:04.682073 28820 consensus_queue.cc:260] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 34693 } } peers { permanent_uuid: "9fe85ae9d98c4fa29991789e257670af" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34671 } } peers { permanent_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39477 } }
I20260929 03:03:04.685026 28821 raft_consensus.cc:348] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 34693 } } peers { permanent_uuid: "9fe85ae9d98c4fa29991789e257670af" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34671 } } peers { permanent_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39477 } }
I20260929 03:03:04.685555 28821 raft_consensus.cc:374] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:04.685676 28821 raft_consensus.cc:729] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fe85ae9d98c4fa29991789e257670af, State: Initialized, Role: FOLLOWER
I20260929 03:03:04.686367 28821 consensus_queue.cc:260] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 34693 } } peers { permanent_uuid: "9fe85ae9d98c4fa29991789e257670af" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34671 } } peers { permanent_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39477 } }
I20260929 03:03:04.690281 28668 heartbeater.cc:503] Master 127.8.208.126:35455 was elected leader, sending a full tablet report...
I20260929 03:03:04.690490 28820 ts_tablet_manager.cc:1428] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1: Time spent starting tablet: real 0.020s user 0.019s sys 0.000s
I20260929 03:03:04.690627 28819 ts_tablet_manager.cc:1428] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e: Time spent starting tablet: real 0.020s user 0.015s sys 0.005s
I20260929 03:03:04.690902 28801 heartbeater.cc:503] Master 127.8.208.126:35455 was elected leader, sending a full tablet report...
I20260929 03:03:04.691672 28825 raft_consensus.cc:482] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:03:04.693616 28821 ts_tablet_manager.cc:1428] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af: Time spent starting tablet: real 0.020s user 0.016s sys 0.004s
I20260929 03:03:04.693396 28825 raft_consensus.cc:504] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 34693 } } peers { permanent_uuid: "9fe85ae9d98c4fa29991789e257670af" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34671 } } peers { permanent_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39477 } }
I20260929 03:03:04.698731 28825 leader_election.cc:288] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fe85ae9d98c4fa29991789e257670af (127.8.208.65:34671), 5f30f1c475a648a4bb3e72ac04a8ddd1 (127.8.208.67:39477)
I20260929 03:03:04.705552 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.697272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:33529 (local address 127.8.208.65:34671)
0929 03:03:04.697466 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:04.697472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:04.698863 (+ 1391us) server_negotiation.cc:407] Connection header received
0929 03:03:04.699265 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:04.699274 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:04.699362 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:04.699422 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:04.700447 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.701534 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.703516 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.704603 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.704770 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.704846 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:04.705158 (+ 312us) server_negotiation.cc:299] Negotiation successful
0929 03:03:04.705306 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 03:03:04.705737 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.696662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:34671 (local address 127.8.208.66:33529)
0929 03:03:04.698747 (+ 2085us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:04.698772 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:04.699067 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:04.699620 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:04.699628 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:04.699665 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:04.700232 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.700249 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.701656 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.701663 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.703353 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.703365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.704812 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.704818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.704982 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.705006 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 03:03:04.705134 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 03:03:04.705288 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1887,"mutex_wait_us":65,"thread_start_us":1642,"threads_started":1}
I20260929 03:03:04.705739 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.699353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:50571 (local address 127.8.208.67:39477)
0929 03:03:04.699534 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:04.699542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:04.699567 (+ 25us) server_negotiation.cc:407] Connection header received
0929 03:03:04.699774 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:04.699779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:04.699846 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:04.699947 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:04.700664 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.701515 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.703761 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.704765 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.704938 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.705013 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:04.705361 (+ 348us) server_negotiation.cc:299] Negotiation successful
0929 03:03:04.705493 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 03:03:04.705886 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.698917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:39477 (local address 127.8.208.66:50571)
0929 03:03:04.699475 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:04.699494 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:04.699653 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:04.699968 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:04.699972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:04.700045 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:04.700510 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.700519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.701646 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.701651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.703438 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:04.703624 (+ 186us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.704997 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.705003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:04.705148 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.705236 (+ 88us) client_negotiation.cc:769] Sending connection context
0929 03:03:04.705337 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 03:03:04.705464 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":33,"thread_start_us":592,"threads_started":1}
I20260929 03:03:04.706739 28489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc2d4cdab3f44738934c4546e232208" candidate_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fe85ae9d98c4fa29991789e257670af" is_pre_election: true
I20260929 03:03:04.706769 28756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc2d4cdab3f44738934c4546e232208" candidate_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1" is_pre_election: true
I20260929 03:03:04.707237 28489 raft_consensus.cc:2454] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eccb1f3a7e74a54913a45a9d2d8101e in term 0.
I20260929 03:03:04.707255 28756 raft_consensus.cc:2454] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eccb1f3a7e74a54913a45a9d2d8101e in term 0.
I20260929 03:03:04.707873 28557 leader_election.cc:302] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f30f1c475a648a4bb3e72ac04a8ddd1, 9eccb1f3a7e74a54913a45a9d2d8101e; no voters:
I20260929 03:03:04.708277 28826 raft_consensus.cc:2781] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:03:04.708381 28826 raft_consensus.cc:482] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:03:04.708478 28826 raft_consensus.cc:3037] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:04.710752 28826 raft_consensus.cc:504] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 34693 } } peers { permanent_uuid: "9fe85ae9d98c4fa29991789e257670af" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34671 } } peers { permanent_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39477 } }
I20260929 03:03:04.711354 28826 leader_election.cc:288] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [CANDIDATE]: Term 1 election: Requested vote from peers 9fe85ae9d98c4fa29991789e257670af (127.8.208.65:34671), 5f30f1c475a648a4bb3e72ac04a8ddd1 (127.8.208.67:39477)
I20260929 03:03:04.711772 28489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc2d4cdab3f44738934c4546e232208" candidate_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fe85ae9d98c4fa29991789e257670af"
I20260929 03:03:04.711962 28489 raft_consensus.cc:3037] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:04.712004 28756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc2d4cdab3f44738934c4546e232208" candidate_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1"
I20260929 03:03:04.712246 28756 raft_consensus.cc:3037] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:04.714265 28489 raft_consensus.cc:2454] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eccb1f3a7e74a54913a45a9d2d8101e in term 1.
I20260929 03:03:04.714524 28756 raft_consensus.cc:2454] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eccb1f3a7e74a54913a45a9d2d8101e in term 1.
I20260929 03:03:04.714687 28556 leader_election.cc:302] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eccb1f3a7e74a54913a45a9d2d8101e, 9fe85ae9d98c4fa29991789e257670af; no voters:
I20260929 03:03:04.714960 28826 raft_consensus.cc:2781] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:04.715106 28826 raft_consensus.cc:686] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 1 LEADER]: Becoming Leader. State: Replica: 9eccb1f3a7e74a54913a45a9d2d8101e, State: Running, Role: LEADER
I20260929 03:03:04.715587 28826 consensus_queue.cc:237] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 34693 } } peers { permanent_uuid: "9fe85ae9d98c4fa29991789e257670af" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34671 } } peers { permanent_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39477 } }
I20260929 03:03:04.720028 28344 catalog_manager.cc:5495] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9eccb1f3a7e74a54913a45a9d2d8101e (127.8.208.66). New cstate: current_term: 1 leader_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eccb1f3a7e74a54913a45a9d2d8101e" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9fe85ae9d98c4fa29991789e257670af" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39477 } health_report { overall_health: UNKNOWN } } }
I20260929 03:03:04.734292 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.729364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53140 (local address 127.8.208.66:34693)
0929 03:03:04.729613 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:04.729617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:04.729634 (+ 17us) server_negotiation.cc:407] Connection header received
0929 03:03:04.729796 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:04.729800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:04.729880 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:04.729994 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:04.730303 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.730991 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.732297 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.732686 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.732777 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.733245 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:04.733742 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:04.733847 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:04.733931 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 03:03:04.733991 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260929 03:03:04.741851 28489 raft_consensus.cc:1261] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af [term 1 FOLLOWER]: Refusing update from remote peer 9eccb1f3a7e74a54913a45a9d2d8101e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:03:04.741851 28756 raft_consensus.cc:1261] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1 [term 1 FOLLOWER]: Refusing update from remote peer 9eccb1f3a7e74a54913a45a9d2d8101e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:03:04.742805 28825 consensus_queue.cc:1035] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f30f1c475a648a4bb3e72ac04a8ddd1" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:03:04.743119 28826 consensus_queue.cc:1035] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fe85ae9d98c4fa29991789e257670af" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 34671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:03:04.754240 28833 mvcc.cc:205] Tried to move back new op lower bound from 7334506433493397504 to 7334506433399050240. Current Snapshot: MvccSnapshot[applied={T|T < 7334506433493397504}]
I20260929 03:03:04.768203 28345 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:03:04.768679 28345 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:03:04.772943 28345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 207f14a70db34b238390c8d90a2d1692: Sending DeleteTablet for 3 replicas of tablet bdc2d4cdab3f44738934c4546e232208
I20260929 03:03:04.774222 28469 tablet_service.cc:1514] Processing DeleteTablet for tablet bdc2d4cdab3f44738934c4546e232208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:03:04 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:57544
I20260929 03:03:04.774228 28603 tablet_service.cc:1514] Processing DeleteTablet for tablet bdc2d4cdab3f44738934c4546e232208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:03:04 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:53124
I20260929 03:03:04.774618 28736 tablet_service.cc:1514] Processing DeleteTablet for tablet bdc2d4cdab3f44738934c4546e232208 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:03:04 UTC) from {username='kudu', principal='kudu/127.8.208.126@KRBTEST.COM'} at 127.0.0.1:36722
I20260929 03:03:04.776635 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 28406
I20260929 03:03:04.786439 28843 tablet_replica.cc:330] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af: stopping tablet replica
I20260929 03:03:04.786723 28844 tablet_replica.cc:330] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e: stopping tablet replica
I20260929 03:03:04.786942 28843 raft_consensus.cc:2229] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:03:04.787204 28844 raft_consensus.cc:2229] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:03:04.787482 28843 raft_consensus.cc:2258] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:03:04.788015 28844 raft_consensus.cc:2258] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:03:04.789803 28843 ts_tablet_manager.cc:1905] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:03:04.791895 28844 ts_tablet_manager.cc:1905] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:03:04.794749 28845 tablet_replica.cc:330] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1: stopping tablet replica
I20260929 03:03:04.795241 28845 raft_consensus.cc:2229] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:03:04.795773 28845 raft_consensus.cc:2258] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:03:04.796905 28843 ts_tablet_manager.cc:1918] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:03:04.797078 28843 log.cc:1196] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/bdc2d4cdab3f44738934c4546e232208
I20260929 03:03:04.797632 28843 ts_tablet_manager.cc:1939] T bdc2d4cdab3f44738934c4546e232208 P 9fe85ae9d98c4fa29991789e257670af: Deleting consensus metadata
I20260929 03:03:04.799302 28844 ts_tablet_manager.cc:1918] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:03:04.799511 28844 log.cc:1196] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/bdc2d4cdab3f44738934c4546e232208
I20260929 03:03:04.799495 28329 catalog_manager.cc:4841] TS 9fe85ae9d98c4fa29991789e257670af (127.8.208.65:34671): tablet bdc2d4cdab3f44738934c4546e232208 (table test-table [id=21d97f71be5a41f1bc8c8c05f0c6363f]) successfully deleted
I20260929 03:03:04.800179 28845 ts_tablet_manager.cc:1905] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:03:04.800485 28844 ts_tablet_manager.cc:1939] T bdc2d4cdab3f44738934c4546e232208 P 9eccb1f3a7e74a54913a45a9d2d8101e: Deleting consensus metadata
I20260929 03:03:04.802199 28331 catalog_manager.cc:4841] TS 9eccb1f3a7e74a54913a45a9d2d8101e (127.8.208.66:34693): tablet bdc2d4cdab3f44738934c4546e232208 (table test-table [id=21d97f71be5a41f1bc8c8c05f0c6363f]) successfully deleted
Sep 29 03:03:04 dist-test-slave-m1rv krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.65@KRBTEST.COM
I20260929 03:03:04.806531 28845 ts_tablet_manager.cc:1918] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:03:04.806700 28845 log.cc:1196] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/bdc2d4cdab3f44738934c4546e232208
I20260929 03:03:04.807144 28845 ts_tablet_manager.cc:1939] T bdc2d4cdab3f44738934c4546e232208 P 5f30f1c475a648a4bb3e72ac04a8ddd1: Deleting consensus metadata
I20260929 03:03:04.808491 28330 catalog_manager.cc:4841] TS 5f30f1c475a648a4bb3e72ac04a8ddd1 (127.8.208.67:39477): tablet bdc2d4cdab3f44738934c4546e232208 (table test-table [id=21d97f71be5a41f1bc8c8c05f0c6363f]) successfully deleted
I20260929 03:03:04.811755 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.778116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57552 (local address 127.8.208.65:34671)
0929 03:03:04.778297 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:04.778303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:04.788862 (+ 10559us) server_negotiation.cc:407] Connection header received
0929 03:03:04.789163 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:04.789210 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:04.789317 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:04.789396 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:04.791327 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.792491 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.796194 (+ 3703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.796896 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.796998 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.804639 (+ 7641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:04.804654 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:04.804672 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:04.804722 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:04.808933 (+ 4211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:04.809688 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:04.809697 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:04.809700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:04.809849 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:04.810203 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:04.810210 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:04.810213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:04.810665 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:04.810958 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:04.811157 (+ 199us) server_negotiation.cc:299] Negotiation successful
0929 03:03:04.811377 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 03:03:04.812510 28529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:04.904277 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 28406
I20260929 03:03:04.922564 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 28540
Sep 29 03:03:04 dist-test-slave-m1rv krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.66@KRBTEST.COM
I20260929 03:03:04.939103 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:04.923396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53146 (local address 127.8.208.66:34693)
0929 03:03:04.923595 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:04.923600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:04.923683 (+ 83us) server_negotiation.cc:407] Connection header received
0929 03:03:04.923840 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:04.923846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:04.923926 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:04.924047 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:04.925418 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.926638 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.927972 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:04.928419 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:04.928591 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:04.932876 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:04.932892 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:04.932911 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:04.932949 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:04.935984 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:04.936689 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:04.936696 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:04.936698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:04.936841 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:04.937474 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:04.937478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:04.937480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:04.937861 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:04.938193 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:04.938473 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 03:03:04.938699 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260929 03:03:04.939779 28663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:05.055308 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 28540
I20260929 03:03:05.071256 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 28673
Sep 29 03:03:05 dist-test-slave-m1rv krb5kdc[28300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.67@KRBTEST.COM
I20260929 03:03:05.085925 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:05.072062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36726 (local address 127.8.208.67:39477)
0929 03:03:05.072217 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:05.072221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:05.072239 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:03:05.072408 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:05.072412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:05.072487 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:05.072570 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:05.073867 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:05.074580 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:05.075802 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:05.076169 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:05.076256 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:05.080512 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:05.080525 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:05.080541 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:05.080575 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:05.083346 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:05.083942 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:05.083950 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:05.083953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:05.084122 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:05.084478 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:05.084482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:05.084484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:05.084822 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:05.085038 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:05.085509 (+ 471us) server_negotiation.cc:299] Negotiation successful
0929 03:03:05.085674 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 03:03:05.086575 28796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:05.174661 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 28673
I20260929 03:03:05.191447 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 28313
I20260929 03:03:05.192551 28375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:05.301625 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 28313
2026-09-29T03:03:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4176 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T03:03:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:05Z Disabled control of system clock
I20260929 03:03:05.375635 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:34221
--webserver_interface=127.8.208.126
--webserver_port=0
--builtin_ntp_servers=127.8.208.84:35155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:34221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 03:03:05.704258 28854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:05.704552 28854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:05.704704 28854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:05.712182 28854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:05.712301 28854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:05.712397 28854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:05.712510 28854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:05.712589 28854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:05.722147 28854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:35155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:34221
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:34221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:05.723479 28854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:05.725310 28854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:05.737221 28859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:05.737118 28862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:05.737118 28860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:05.738582 28854 server_base.cc:1034] running on GCE node
I20260929 03:03:05.739346 28854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:05.740743 28854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:05.741942 28854 hybrid_clock.cc:648] HybridClock initialized: now 1790650985741887 us; error 46 us; skew 500 ppm
I20260929 03:03:05.742268 28854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 03:03:05.744460 28854 webserver.cc:460] Webserver started at http://127.8.208.126:41461/ using document root <none> and password file <none>
I20260929 03:03:05.745121 28854 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:05.745270 28854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:05.745555 28854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:05.748373 28854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "70cb1b6d4f3b46339d60b5819dfe9d56"
format_stamp: "Formatted at 2026-09-29 03:03:05 on dist-test-slave-m1rv"
server_key: "c52f29dfa65c06b5602915acdd30c2c8"
server_key_iv: "fc362acd97a552c6994c31418ca372a1"
server_key_version: "encryptionkey@0"
I20260929 03:03:05.749125 28854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "70cb1b6d4f3b46339d60b5819dfe9d56"
format_stamp: "Formatted at 2026-09-29 03:03:05 on dist-test-slave-m1rv"
server_key: "c52f29dfa65c06b5602915acdd30c2c8"
server_key_iv: "fc362acd97a552c6994c31418ca372a1"
server_key_version: "encryptionkey@0"
I20260929 03:03:05.754109 28854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 03:03:05.757516 28868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:05.758925 28854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 03:03:05.759086 28854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "70cb1b6d4f3b46339d60b5819dfe9d56"
format_stamp: "Formatted at 2026-09-29 03:03:05 on dist-test-slave-m1rv"
server_key: "c52f29dfa65c06b5602915acdd30c2c8"
server_key_iv: "fc362acd97a552c6994c31418ca372a1"
server_key_version: "encryptionkey@0"
I20260929 03:03:05.759258 28854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:05.773720 28854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:05.774494 28854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:05.774741 28854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:05.793965 28854 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:34221
I20260929 03:03:05.794018 28919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:34221 every 8 connection(s)
I20260929 03:03:05.795437 28854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:05.797423 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 28854
I20260929 03:03:05.797629 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:05.798197 9025 external_mini_cluster.cc:1428] Setting key ef0503f58c762c9f4a033f86f71ae8e2
I20260929 03:03:05.799934 28920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:05.811477 28923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:05.800189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41726 (local address 127.8.208.126:34221)
0929 03:03:05.801147 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:05.801157 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:05.801258 (+ 101us) server_negotiation.cc:407] Connection header received
0929 03:03:05.802819 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:05.802840 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:05.803293 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:05.803694 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:05.804251 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:05.805691 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:05.807053 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:05.807709 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:05.807906 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:05.809128 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:05.809135 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:05.809150 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:05.809787 (+ 637us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:05.809861 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:05.809881 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:05.810030 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:05.810248 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 03:03:05.810556 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":342,"threads_started":1}
I20260929 03:03:05.811709 28920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56: Bootstrap starting.
I20260929 03:03:05.815189 28920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:05.816167 28920 log.cc:824] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:05.818823 28920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56: No bootstrap required, opened a new log
I20260929 03:03:05.826251 28920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cb1b6d4f3b46339d60b5819dfe9d56" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34221 } }
I20260929 03:03:05.826618 28920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:05.826743 28920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70cb1b6d4f3b46339d60b5819dfe9d56, State: Initialized, Role: FOLLOWER
I20260929 03:03:05.827428 28920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cb1b6d4f3b46339d60b5819dfe9d56" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34221 } }
I20260929 03:03:05.827705 28920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:05.827881 28920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:05.828075 28920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:05.830286 28920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cb1b6d4f3b46339d60b5819dfe9d56" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34221 } }
I20260929 03:03:05.830780 28920 leader_election.cc:302] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70cb1b6d4f3b46339d60b5819dfe9d56; no voters:
I20260929 03:03:05.831348 28920 leader_election.cc:288] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:05.831519 28925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:05.832305 28925 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [term 1 LEADER]: Becoming Leader. State: Replica: 70cb1b6d4f3b46339d60b5819dfe9d56, State: Running, Role: LEADER
I20260929 03:03:05.832903 28925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cb1b6d4f3b46339d60b5819dfe9d56" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34221 } }
I20260929 03:03:05.833413 28920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:05.835500 28926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70cb1b6d4f3b46339d60b5819dfe9d56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cb1b6d4f3b46339d60b5819dfe9d56" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34221 } } }
I20260929 03:03:05.835948 28926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:05.836980 28932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:05.837009 28927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70cb1b6d4f3b46339d60b5819dfe9d56. Latest consensus state: current_term: 1 leader_uuid: "70cb1b6d4f3b46339d60b5819dfe9d56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cb1b6d4f3b46339d60b5819dfe9d56" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34221 } } }
I20260929 03:03:05.838330 28927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:05.843606 28932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:05.852402 28932 catalog_manager.cc:1347] Generated new cluster ID: 12a2adc42534490b82dafb6e245b65e0
I20260929 03:03:05.852543 28932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:05.889657 28932 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:05.890961 28932 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:05.901554 28932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56: Generated new TSK 0
I20260929 03:03:05.902408 28932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:05.913388 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.65:0
--local_ip_for_outbound_sockets=127.8.208.65
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:34221
--builtin_ntp_servers=127.8.208.84:35155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 03:03:06.258782 28944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:06.259104 28944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:06.259302 28944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:06.267845 28944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:06.267982 28944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:06.268208 28944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.65
I20260929 03:03:06.278333 28944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:35155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.208.65
--webserver_port=0
--tserver_master_addrs=127.8.208.126:34221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.65
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:06.279711 28944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:06.281564 28944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:06.294838 28949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:06.294816 28952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:06.294909 28950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:06.296396 28944 server_base.cc:1034] running on GCE node
I20260929 03:03:06.297132 28944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:06.298208 28944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:06.299413 28944 hybrid_clock.cc:648] HybridClock initialized: now 1790650986299370 us; error 32 us; skew 500 ppm
I20260929 03:03:06.299744 28944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 03:03:06.302068 28944 webserver.cc:460] Webserver started at http://127.8.208.65:39975/ using document root <none> and password file <none>
I20260929 03:03:06.302766 28944 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:06.302901 28944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:06.303236 28944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:06.306165 28944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/data/instance:
uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b"
format_stamp: "Formatted at 2026-09-29 03:03:06 on dist-test-slave-m1rv"
server_key: "986db943c4eed5eb59c56a8c19d25411"
server_key_iv: "f05492f3cb2b33eec1285c41274ab4cf"
server_key_version: "encryptionkey@0"
I20260929 03:03:06.306924 28944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance:
uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b"
format_stamp: "Formatted at 2026-09-29 03:03:06 on dist-test-slave-m1rv"
server_key: "986db943c4eed5eb59c56a8c19d25411"
server_key_iv: "f05492f3cb2b33eec1285c41274ab4cf"
server_key_version: "encryptionkey@0"
I20260929 03:03:06.312000 28944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 03:03:06.315492 28958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:06.316839 28944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260929 03:03:06.317042 28944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/wal
uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b"
format_stamp: "Formatted at 2026-09-29 03:03:06 on dist-test-slave-m1rv"
server_key: "986db943c4eed5eb59c56a8c19d25411"
server_key_iv: "f05492f3cb2b33eec1285c41274ab4cf"
server_key_version: "encryptionkey@0"
I20260929 03:03:06.317308 28944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:06.337524 28944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:06.338459 28944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:06.338809 28944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:06.339861 28944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:03:06.341401 28944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:03:06.341487 28944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:06.341656 28944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:03:06.341756 28944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:06.375387 28944 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.65:44729
I20260929 03:03:06.375480 29070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.65:44729 every 8 connection(s)
I20260929 03:03:06.376703 28944 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/data/info.pb
I20260929 03:03:06.378300 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 28944
I20260929 03:03:06.378474 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/wal/instance
I20260929 03:03:06.378979 9025 external_mini_cluster.cc:1428] Setting key b2479369eec4ffc173ef40a633f87e3b
I20260929 03:03:06.382299 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.66:0
--local_ip_for_outbound_sockets=127.8.208.66
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:34221
--builtin_ntp_servers=127.8.208.84:35155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 03:03:06.392843 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:06.379495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:59447 (local address 127.8.208.126:34221)
0929 03:03:06.380035 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:06.380043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:06.381517 (+ 1474us) server_negotiation.cc:407] Connection header received
0929 03:03:06.384406 (+ 2889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:06.384413 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:06.384479 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:06.384578 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:06.386022 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:06.386765 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:06.389288 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:06.389692 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:06.389791 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:06.391942 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:06.391952 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:06.391957 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:06.391985 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:06.392038 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:06.392056 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:06.392150 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:06.392424 (+ 274us) server_negotiation.cc:299] Negotiation successful
0929 03:03:06.392534 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":275,"threads_started":1}
I20260929 03:03:06.394152 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:06.380179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:34221 (local address 127.8.208.65:59447)
0929 03:03:06.381342 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:06.381393 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:06.384079 (+ 2686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:06.384851 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:06.384867 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:06.384993 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:06.385832 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:06.385859 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:06.386931 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:06.386939 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:06.389108 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:06.389126 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:06.389924 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:06.389934 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:06.390276 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:06.391456 (+ 1180us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:03:06.391462 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:06.391526 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:03:06.391654 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:06.392170 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:06.392179 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:06.392185 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 03:03:06.392321 (+ 136us) client_negotiation.cc:240] Negotiation successful
0929 03:03:06.392736 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":352,"threads_started":1}
I20260929 03:03:06.395977 29071 heartbeater.cc:347] Connected to a master server at 127.8.208.126:34221
I20260929 03:03:06.396310 29071 heartbeater.cc:464] Registering TS with master...
I20260929 03:03:06.397131 29071 heartbeater.cc:511] Master 127.8.208.126:34221 requested a full tablet report, sending...
I20260929 03:03:06.399479 28885 ts_manager.cc:194] Registered new tserver with Master: 0bdab4cad8c74ccf9b9eacc723f5d94b (127.8.208.65:44729)
I20260929 03:03:06.401762 28885 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.208.65:59447
W20260929 03:03:06.709456 29075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:06.709753 29075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:06.709910 29075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:06.717567 29075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:06.717670 29075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:06.717774 29075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.66
I20260929 03:03:06.727613 29075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:35155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.208.66
--webserver_port=0
--tserver_master_addrs=127.8.208.126:34221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.66
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:06.728929 29075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:06.730844 29075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:06.745055 29081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:06.745160 29083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:06.745083 29080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:06.746179 29075 server_base.cc:1034] running on GCE node
I20260929 03:03:06.746758 29075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:06.747761 29075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:06.749033 29075 hybrid_clock.cc:648] HybridClock initialized: now 1790650986748986 us; error 39 us; skew 500 ppm
I20260929 03:03:06.749421 29075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 03:03:06.751852 29075 webserver.cc:460] Webserver started at http://127.8.208.66:37935/ using document root <none> and password file <none>
I20260929 03:03:06.752595 29075 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:06.752724 29075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:06.753087 29075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:06.756232 29075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/data/instance:
uuid: "b5c81483a1a14f1793f1ae0247374fa3"
format_stamp: "Formatted at 2026-09-29 03:03:06 on dist-test-slave-m1rv"
server_key: "6ecb8867e85d9669701cbb16cafc8d50"
server_key_iv: "0c5106cea2ff2184e40b700529c6f494"
server_key_version: "encryptionkey@0"
I20260929 03:03:06.757113 29075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance:
uuid: "b5c81483a1a14f1793f1ae0247374fa3"
format_stamp: "Formatted at 2026-09-29 03:03:06 on dist-test-slave-m1rv"
server_key: "6ecb8867e85d9669701cbb16cafc8d50"
server_key_iv: "0c5106cea2ff2184e40b700529c6f494"
server_key_version: "encryptionkey@0"
I20260929 03:03:06.762471 29075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 03:03:06.766219 29089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:06.767562 29075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:03:06.767751 29075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/wal
uuid: "b5c81483a1a14f1793f1ae0247374fa3"
format_stamp: "Formatted at 2026-09-29 03:03:06 on dist-test-slave-m1rv"
server_key: "6ecb8867e85d9669701cbb16cafc8d50"
server_key_iv: "0c5106cea2ff2184e40b700529c6f494"
server_key_version: "encryptionkey@0"
I20260929 03:03:06.767997 29075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:06.787662 29075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:06.788532 29075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:06.788857 29075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:06.789860 29075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:03:06.791312 29075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:03:06.791396 29075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:06.791564 29075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:03:06.791662 29075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:06.828040 29075 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.66:33827
I20260929 03:03:06.828086 29201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.66:33827 every 8 connection(s)
I20260929 03:03:06.829403 29075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/data/info.pb
I20260929 03:03:06.832073 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 29075
I20260929 03:03:06.832260 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/wal/instance
I20260929 03:03:06.832676 9025 external_mini_cluster.cc:1428] Setting key 44e1a24dc277bc435a36913ce0d6a77a
I20260929 03:03:06.836041 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.208.67:0
--local_ip_for_outbound_sockets=127.8.208.67
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:34221
--builtin_ntp_servers=127.8.208.84:35155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 03:03:06.844156 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:06.831960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.66:42619 (local address 127.8.208.126:34221)
0929 03:03:06.832174 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:06.832181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:06.833882 (+ 1701us) server_negotiation.cc:407] Connection header received
0929 03:03:06.836083 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:06.836088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:06.836197 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:06.836305 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:06.837771 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:06.838571 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:06.841001 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:06.841467 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:06.841565 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:06.843244 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:06.843253 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:06.843260 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:06.843285 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:06.843324 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:06.843340 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:06.843431 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:06.843683 (+ 252us) server_negotiation.cc:299] Negotiation successful
0929 03:03:06.843788 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 03:03:06.845471 29204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:06.832559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:34221 (local address 127.8.208.66:42619)
0929 03:03:06.833646 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:06.833694 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:06.835796 (+ 2102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:06.836555 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:06.836569 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:06.836710 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:06.837592 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:06.837620 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:06.838699 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:06.838706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:06.840836 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:06.840854 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:06.841688 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:06.841698 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:06.842060 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:06.842841 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:03:06.842847 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:06.842904 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:03:06.843022 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:06.843450 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:06.843463 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:06.843473 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 03:03:06.843660 (+ 187us) client_negotiation.cc:240] Negotiation successful
0929 03:03:06.844104 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":315,"threads_started":1}
I20260929 03:03:06.846889 29202 heartbeater.cc:347] Connected to a master server at 127.8.208.126:34221
I20260929 03:03:06.847200 29202 heartbeater.cc:464] Registering TS with master...
I20260929 03:03:06.847903 29202 heartbeater.cc:511] Master 127.8.208.126:34221 requested a full tablet report, sending...
I20260929 03:03:06.849282 28885 ts_manager.cc:194] Registered new tserver with Master: b5c81483a1a14f1793f1ae0247374fa3 (127.8.208.66:33827)
I20260929 03:03:06.850062 28885 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.208.66:42619
W20260929 03:03:07.163254 29205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:07.163529 29205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:07.163708 29205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:07.171128 29205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:07.171226 29205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:07.171406 29205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.208.67
I20260929 03:03:07.180404 29205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:35155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.208.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.208.67
--webserver_port=0
--tserver_master_addrs=127.8.208.126:34221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.208.67
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:07.181699 29205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:07.183465 29205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:07.196614 29211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:07.196662 29213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:07.196695 29210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:07.197425 29205 server_base.cc:1034] running on GCE node
I20260929 03:03:07.198043 29205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:07.198995 29205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:07.200241 29205 hybrid_clock.cc:648] HybridClock initialized: now 1790650987200204 us; error 49 us; skew 500 ppm
I20260929 03:03:07.200578 29205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 03:03:07.202919 29205 webserver.cc:460] Webserver started at http://127.8.208.67:34731/ using document root <none> and password file <none>
I20260929 03:03:07.203647 29205 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:07.203787 29205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:07.204123 29205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:07.207070 29205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/data/instance:
uuid: "4083bcc2da714c8f862a7ac5a3c64e8e"
format_stamp: "Formatted at 2026-09-29 03:03:07 on dist-test-slave-m1rv"
server_key: "93db2c36392a95467b718f266063f1fd"
server_key_iv: "1302f6e0abaab8bc00d70ad9103bac70"
server_key_version: "encryptionkey@0"
I20260929 03:03:07.207870 29205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance:
uuid: "4083bcc2da714c8f862a7ac5a3c64e8e"
format_stamp: "Formatted at 2026-09-29 03:03:07 on dist-test-slave-m1rv"
server_key: "93db2c36392a95467b718f266063f1fd"
server_key_iv: "1302f6e0abaab8bc00d70ad9103bac70"
server_key_version: "encryptionkey@0"
I20260929 03:03:07.212893 29205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 03:03:07.216526 29219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:07.217927 29205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:03:07.218146 29205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/wal
uuid: "4083bcc2da714c8f862a7ac5a3c64e8e"
format_stamp: "Formatted at 2026-09-29 03:03:07 on dist-test-slave-m1rv"
server_key: "93db2c36392a95467b718f266063f1fd"
server_key_iv: "1302f6e0abaab8bc00d70ad9103bac70"
server_key_version: "encryptionkey@0"
I20260929 03:03:07.218374 29205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:07.236002 29205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:07.236855 29205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:07.237221 29205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:07.238245 29205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:03:07.239873 29205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:03:07.240003 29205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:07.240134 29205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:03:07.240232 29205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:07.273423 29205 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.67:40951
I20260929 03:03:07.273515 29331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.67:40951 every 8 connection(s)
I20260929 03:03:07.274912 29205 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/data/info.pb
I20260929 03:03:07.282433 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 29205
I20260929 03:03:07.282634 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/wal/instance
I20260929 03:03:07.283102 9025 external_mini_cluster.cc:1428] Setting key b9f1061c1300bf6c515ba50c4a49dbd7
I20260929 03:03:07.288224 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.277694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.67:49319 (local address 127.8.208.126:34221)
0929 03:03:07.277878 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:07.277882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:07.279282 (+ 1400us) server_negotiation.cc:407] Connection header received
0929 03:03:07.280966 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:07.280970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:07.281042 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:07.281137 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:07.282490 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.283369 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.285530 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.286140 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.286240 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.287557 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:07.287562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:07.287565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:07.287583 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:07.287614 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:07.287625 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:07.287693 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:07.287881 (+ 188us) server_negotiation.cc:299] Negotiation successful
0929 03:03:07.287957 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 03:03:07.289212 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.278232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:34221 (local address 127.8.208.67:49319)
0929 03:03:07.279110 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:07.279151 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:07.280700 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:07.281444 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:07.281452 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:07.281560 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:07.282273 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:07.282299 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.283542 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.283546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:07.285400 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:07.285412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.286327 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.286332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:07.286541 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.287273 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:03:07.287276 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:07.287316 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:03:07.287395 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:07.287708 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:07.287716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:07.287721 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 03:03:07.287854 (+ 133us) client_negotiation.cc:240] Negotiation successful
0929 03:03:07.288142 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":286,"threads_started":1}
I20260929 03:03:07.290680 29332 heartbeater.cc:347] Connected to a master server at 127.8.208.126:34221
I20260929 03:03:07.290927 29332 heartbeater.cc:464] Registering TS with master...
I20260929 03:03:07.291586 29332 heartbeater.cc:511] Master 127.8.208.126:34221 requested a full tablet report, sending...
I20260929 03:03:07.292865 28885 ts_manager.cc:194] Registered new tserver with Master: 4083bcc2da714c8f862a7ac5a3c64e8e (127.8.208.67:40951)
I20260929 03:03:07.293725 28885 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.208.67:49319
I20260929 03:03:07.298002 9025 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:03:07.308185 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.302309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41742 (local address 127.8.208.126:34221)
0929 03:03:07.302576 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:07.302583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:07.302813 (+ 230us) server_negotiation.cc:407] Connection header received
0929 03:03:07.303075 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:07.303086 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:07.303159 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:07.303270 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:07.303754 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.304448 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.305856 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.306298 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.306445 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.307481 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:07.307486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:07.307489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:07.307505 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:07.307535 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:07.307547 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:07.307640 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:07.307794 (+ 154us) server_negotiation.cc:299] Negotiation successful
0929 03:03:07.307882 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 03:03:07.311794 28885 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:03:07.318629 28885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:03:07.354693 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.331872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50650 (local address 127.8.208.65:44729)
0929 03:03:07.345945 (+ 14073us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:07.345955 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:07.346477 (+ 522us) server_negotiation.cc:407] Connection header received
0929 03:03:07.346770 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:07.346777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:07.346947 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:07.347149 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:07.348596 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.349968 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.351922 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.353392 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.353540 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.353814 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:07.354150 (+ 336us) server_negotiation.cc:299] Negotiation successful
0929 03:03:07.354322 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13913,"thread_start_us":13785,"threads_started":1}
I20260929 03:03:07.354738 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.329941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:40951 (local address 127.0.0.1:33622)
0929 03:03:07.330999 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:07.331144 (+ 145us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:07.331315 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:07.347189 (+ 15874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:07.347199 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:07.347258 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:07.347871 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:07.347884 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.349417 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.349423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:07.351959 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:07.351978 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.353772 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.353778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:07.353944 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.353971 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 03:03:07.354067 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 03:03:07.354303 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":191,"spinlock_wait_cycles":11008,"thread_start_us":685,"threads_started":1}
I20260929 03:03:07.355005 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.330254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:33827 (local address 127.0.0.1:45908)
0929 03:03:07.345002 (+ 14748us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:07.345029 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:07.345381 (+ 352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:07.347193 (+ 1812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:07.347198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:07.347243 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:07.347774 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:07.347792 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.349730 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.349735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:07.352062 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:07.352076 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.353897 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.353903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:07.354150 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.354182 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 03:03:07.354240 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 03:03:07.354348 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14535,"mutex_wait_us":24,"spinlock_wait_cycles":3456,"thread_start_us":16033,"threads_started":1}
I20260929 03:03:07.355257 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.330893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45908 (local address 127.8.208.66:33827)
0929 03:03:07.346512 (+ 15619us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:07.346526 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:07.346552 (+ 26us) server_negotiation.cc:407] Connection header received
0929 03:03:07.346631 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:07.346637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:07.346948 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:07.347556 (+ 608us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:07.347986 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.349594 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.352278 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.353646 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.354480 (+ 834us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.354715 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:07.354783 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 03:03:07.354962 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15460,"thread_start_us":15169,"threads_started":1}
I20260929 03:03:07.355459 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.331381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.65:44729 (local address 127.0.0.1:50650)
0929 03:03:07.346341 (+ 14960us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:07.346365 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:07.346583 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:07.347453 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:07.347457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:07.347479 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:07.348328 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:07.348340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.350105 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.350110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:07.351732 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:07.351743 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.353580 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.353587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:07.353802 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.353869 (+ 67us) client_negotiation.cc:769] Sending connection context
0929 03:03:07.354080 (+ 211us) client_negotiation.cc:240] Negotiation successful
0929 03:03:07.354296 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14829,"mutex_wait_us":9,"thread_start_us":13815,"threads_started":1}
I20260929 03:03:07.356986 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.330252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33622 (local address 127.8.208.67:40951)
0929 03:03:07.346539 (+ 16287us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:07.346547 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:07.346576 (+ 29us) server_negotiation.cc:407] Connection header received
0929 03:03:07.346679 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:07.346688 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:07.346947 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:07.347152 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:07.348107 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.349290 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.352172 (+ 2882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.353561 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.355294 (+ 1733us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.356368 (+ 1074us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:07.356437 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 03:03:07.356635 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16117,"thread_start_us":16427,"threads_started":1}
I20260929 03:03:07.358843 29006 tablet_service.cc:1467] Processing CreateTablet for tablet 74ea0dbe38e143eab12eadac3172ac45 (DEFAULT_TABLE table=test-table [id=0a84563f60a147ef884c9c9d097b0475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:03:07.358852 29137 tablet_service.cc:1467] Processing CreateTablet for tablet 74ea0dbe38e143eab12eadac3172ac45 (DEFAULT_TABLE table=test-table [id=0a84563f60a147ef884c9c9d097b0475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:03:07.360226 29137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74ea0dbe38e143eab12eadac3172ac45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:07.360232 29006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74ea0dbe38e143eab12eadac3172ac45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:07.360324 29267 tablet_service.cc:1467] Processing CreateTablet for tablet 74ea0dbe38e143eab12eadac3172ac45 (DEFAULT_TABLE table=test-table [id=0a84563f60a147ef884c9c9d097b0475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:03:07.361622 29267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74ea0dbe38e143eab12eadac3172ac45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:07.371517 29350 tablet_bootstrap.cc:492] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b: Bootstrap starting.
I20260929 03:03:07.374337 29351 tablet_bootstrap.cc:492] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3: Bootstrap starting.
I20260929 03:03:07.375150 29350 tablet_bootstrap.cc:654] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:07.376591 29350 log.cc:824] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:07.378078 29351 tablet_bootstrap.cc:654] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:07.379474 29350 tablet_bootstrap.cc:492] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b: No bootstrap required, opened a new log
I20260929 03:03:07.379638 29351 log.cc:824] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:07.379766 29350 ts_tablet_manager.cc:1397] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260929 03:03:07.384127 29352 tablet_bootstrap.cc:492] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e: Bootstrap starting.
I20260929 03:03:07.389786 29352 tablet_bootstrap.cc:654] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:07.391052 29352 log.cc:824] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:07.391817 29351 tablet_bootstrap.cc:492] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3: No bootstrap required, opened a new log
I20260929 03:03:07.392133 29351 ts_tablet_manager.cc:1397] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.004s
I20260929 03:03:07.391911 29350 raft_consensus.cc:348] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40951 } } peers { permanent_uuid: "b5c81483a1a14f1793f1ae0247374fa3" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 33827 } } peers { permanent_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44729 } }
I20260929 03:03:07.392378 29350 raft_consensus.cc:374] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:07.392501 29350 raft_consensus.cc:729] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdab4cad8c74ccf9b9eacc723f5d94b, State: Initialized, Role: FOLLOWER
I20260929 03:03:07.393118 29350 consensus_queue.cc:260] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40951 } } peers { permanent_uuid: "b5c81483a1a14f1793f1ae0247374fa3" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 33827 } } peers { permanent_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44729 } }
I20260929 03:03:07.394052 29352 tablet_bootstrap.cc:492] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e: No bootstrap required, opened a new log
I20260929 03:03:07.394248 29071 heartbeater.cc:503] Master 127.8.208.126:34221 was elected leader, sending a full tablet report...
I20260929 03:03:07.394332 29352 ts_tablet_manager.cc:1397] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.002s
I20260929 03:03:07.394836 29350 ts_tablet_manager.cc:1428] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260929 03:03:07.401875 29351 raft_consensus.cc:348] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40951 } } peers { permanent_uuid: "b5c81483a1a14f1793f1ae0247374fa3" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 33827 } } peers { permanent_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44729 } }
I20260929 03:03:07.402226 29351 raft_consensus.cc:374] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:07.402370 29351 raft_consensus.cc:729] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5c81483a1a14f1793f1ae0247374fa3, State: Initialized, Role: FOLLOWER
I20260929 03:03:07.402661 29352 raft_consensus.cc:348] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40951 } } peers { permanent_uuid: "b5c81483a1a14f1793f1ae0247374fa3" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 33827 } } peers { permanent_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44729 } }
I20260929 03:03:07.403005 29352 raft_consensus.cc:374] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:07.403127 29352 raft_consensus.cc:729] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083bcc2da714c8f862a7ac5a3c64e8e, State: Initialized, Role: FOLLOWER
I20260929 03:03:07.403009 29351 consensus_queue.cc:260] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40951 } } peers { permanent_uuid: "b5c81483a1a14f1793f1ae0247374fa3" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 33827 } } peers { permanent_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44729 } }
I20260929 03:03:07.403704 29352 consensus_queue.cc:260] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40951 } } peers { permanent_uuid: "b5c81483a1a14f1793f1ae0247374fa3" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 33827 } } peers { permanent_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44729 } }
I20260929 03:03:07.404191 29202 heartbeater.cc:503] Master 127.8.208.126:34221 was elected leader, sending a full tablet report...
I20260929 03:03:07.404786 29351 ts_tablet_manager.cc:1428] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260929 03:03:07.405370 29352 ts_tablet_manager.cc:1428] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260929 03:03:07.405666 29332 heartbeater.cc:503] Master 127.8.208.126:34221 was elected leader, sending a full tablet report...
I20260929 03:03:07.484786 29356 raft_consensus.cc:482] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:03:07.485131 29356 raft_consensus.cc:504] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40951 } } peers { permanent_uuid: "b5c81483a1a14f1793f1ae0247374fa3" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 33827 } } peers { permanent_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44729 } }
I20260929 03:03:07.486855 29356 leader_election.cc:288] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4083bcc2da714c8f862a7ac5a3c64e8e (127.8.208.67:40951), b5c81483a1a14f1793f1ae0247374fa3 (127.8.208.66:33827)
I20260929 03:03:07.494071 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.487513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:54155 (local address 127.8.208.66:33827)
0929 03:03:07.487745 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:07.487754 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:07.488267 (+ 513us) server_negotiation.cc:407] Connection header received
0929 03:03:07.488520 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:07.488526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:07.488648 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:07.488717 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:07.489628 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.490381 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.492370 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.493115 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.493341 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.493410 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:07.493709 (+ 299us) server_negotiation.cc:299] Negotiation successful
0929 03:03:07.493830 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 03:03:07.494128 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.487539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.208.65:34827 (local address 127.8.208.67:40951)
0929 03:03:07.487818 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:07.487827 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:07.488364 (+ 537us) server_negotiation.cc:407] Connection header received
0929 03:03:07.488548 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:07.488554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:07.488642 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:07.488750 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:03:07.489608 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.490388 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.492436 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.493265 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.493435 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.493498 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:07.493787 (+ 289us) server_negotiation.cc:299] Negotiation successful
0929 03:03:07.493903 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260929 03:03:07.494179 29360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.487312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.66:33827 (local address 127.8.208.65:54155)
0929 03:03:07.488135 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:07.488161 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:07.488329 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:07.488927 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:07.488932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:07.488984 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:07.489442 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:07.489453 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.490570 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.490575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:07.492180 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:07.492192 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.493368 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.493374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:07.493525 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.493567 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 03:03:07.493672 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 03:03:07.493802 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":48,"thread_start_us":605,"threads_started":1}
I20260929 03:03:07.494298 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.487322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.67:40951 (local address 127.8.208.65:34827)
0929 03:03:07.488263 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:07.488299 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:07.488400 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:07.488775 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:07.488780 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:07.488806 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:03:07.489370 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:07.489384 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.490513 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.490519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:07.492181 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:07.492192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.493446 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.493452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:07.493630 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.493652 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 03:03:07.493764 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 03:03:07.493877 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":117,"spinlock_wait_cycles":16128,"thread_start_us":497,"threads_started":1}
I20260929 03:03:07.495213 29157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74ea0dbe38e143eab12eadac3172ac45" candidate_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5c81483a1a14f1793f1ae0247374fa3" is_pre_election: true
I20260929 03:03:07.495214 29287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74ea0dbe38e143eab12eadac3172ac45" candidate_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e" is_pre_election: true
I20260929 03:03:07.495693 29157 raft_consensus.cc:2454] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdab4cad8c74ccf9b9eacc723f5d94b in term 0.
I20260929 03:03:07.495720 29287 raft_consensus.cc:2454] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdab4cad8c74ccf9b9eacc723f5d94b in term 0.
I20260929 03:03:07.496341 28959 leader_election.cc:302] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0bdab4cad8c74ccf9b9eacc723f5d94b, 4083bcc2da714c8f862a7ac5a3c64e8e, b5c81483a1a14f1793f1ae0247374fa3; no voters:
I20260929 03:03:07.496731 29356 raft_consensus.cc:2781] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:03:07.496884 29356 raft_consensus.cc:482] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:03:07.497011 29356 raft_consensus.cc:3037] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:07.499300 29356 raft_consensus.cc:504] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40951 } } peers { permanent_uuid: "b5c81483a1a14f1793f1ae0247374fa3" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 33827 } } peers { permanent_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44729 } }
I20260929 03:03:07.499912 29356 leader_election.cc:288] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [CANDIDATE]: Term 1 election: Requested vote from peers 4083bcc2da714c8f862a7ac5a3c64e8e (127.8.208.67:40951), b5c81483a1a14f1793f1ae0247374fa3 (127.8.208.66:33827)
I20260929 03:03:07.500403 29157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74ea0dbe38e143eab12eadac3172ac45" candidate_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5c81483a1a14f1793f1ae0247374fa3"
I20260929 03:03:07.500403 29287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74ea0dbe38e143eab12eadac3172ac45" candidate_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e"
I20260929 03:03:07.500602 29157 raft_consensus.cc:3037] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:07.500619 29287 raft_consensus.cc:3037] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:07.503887 29287 raft_consensus.cc:2454] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdab4cad8c74ccf9b9eacc723f5d94b in term 1.
I20260929 03:03:07.503890 29157 raft_consensus.cc:2454] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdab4cad8c74ccf9b9eacc723f5d94b in term 1.
I20260929 03:03:07.504319 28959 leader_election.cc:302] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdab4cad8c74ccf9b9eacc723f5d94b, 4083bcc2da714c8f862a7ac5a3c64e8e; no voters:
I20260929 03:03:07.504640 29356 raft_consensus.cc:2781] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:07.505105 29356 raft_consensus.cc:686] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 1 LEADER]: Becoming Leader. State: Replica: 0bdab4cad8c74ccf9b9eacc723f5d94b, State: Running, Role: LEADER
I20260929 03:03:07.505591 29356 consensus_queue.cc:237] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40951 } } peers { permanent_uuid: "b5c81483a1a14f1793f1ae0247374fa3" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 33827 } } peers { permanent_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44729 } }
I20260929 03:03:07.509917 28885 catalog_manager.cc:5495] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bdab4cad8c74ccf9b9eacc723f5d94b (127.8.208.65). New cstate: current_term: 1 leader_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5c81483a1a14f1793f1ae0247374fa3" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdab4cad8c74ccf9b9eacc723f5d94b" member_type: VOTER last_known_addr { host: "127.8.208.65" port: 44729 } health_report { overall_health: HEALTHY } } }
W20260929 03:03:07.526837 29333 tablet.cc:2369] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:03:07.538317 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.533040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.8.208.65:44729)
0929 03:03:07.533284 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:07.533289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:07.533309 (+ 20us) server_negotiation.cc:407] Connection header received
0929 03:03:07.533535 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:07.533540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:07.533633 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:07.533762 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:07.534148 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.534922 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.536274 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.536687 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.536781 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.537249 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:07.537788 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:07.537901 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:07.538048 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 03:03:07.538107 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 03:03:07.546787 29157 raft_consensus.cc:1261] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3 [term 1 FOLLOWER]: Refusing update from remote peer 0bdab4cad8c74ccf9b9eacc723f5d94b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:03:07.546787 29287 raft_consensus.cc:1261] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e [term 1 FOLLOWER]: Refusing update from remote peer 0bdab4cad8c74ccf9b9eacc723f5d94b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:03:07.547844 29361 consensus_queue.cc:1035] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5c81483a1a14f1793f1ae0247374fa3" member_type: VOTER last_known_addr { host: "127.8.208.66" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:03:07.548266 29356 consensus_queue.cc:1035] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083bcc2da714c8f862a7ac5a3c64e8e" member_type: VOTER last_known_addr { host: "127.8.208.67" port: 40951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:03:07.564844 29365 mvcc.cc:205] Tried to move back new op lower bound from 7334506444980822016 to 7334506444826439680. Current Snapshot: MvccSnapshot[applied={T|T < 7334506444980822016}]
I20260929 03:03:07.575807 28885 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:03:07.576119 28885 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:03:07.579751 28885 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70cb1b6d4f3b46339d60b5819dfe9d56: Sending DeleteTablet for 3 replicas of tablet 74ea0dbe38e143eab12eadac3172ac45
I20260929 03:03:07.580904 29137 tablet_service.cc:1514] Processing DeleteTablet for tablet 74ea0dbe38e143eab12eadac3172ac45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:03:07 UTC) from {username='slave'} at 127.0.0.1:45908
I20260929 03:03:07.581207 29006 tablet_service.cc:1514] Processing DeleteTablet for tablet 74ea0dbe38e143eab12eadac3172ac45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:03:07 UTC) from {username='slave'} at 127.0.0.1:50650
I20260929 03:03:07.581243 29267 tablet_service.cc:1514] Processing DeleteTablet for tablet 74ea0dbe38e143eab12eadac3172ac45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:03:07 UTC) from {username='slave'} at 127.0.0.1:33622
I20260929 03:03:07.582677 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 28944
I20260929 03:03:07.590939 29375 tablet_replica.cc:330] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b: stopping tablet replica
I20260929 03:03:07.591385 29375 raft_consensus.cc:2229] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:03:07.592229 29375 raft_consensus.cc:2258] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:03:07.597041 29375 ts_tablet_manager.cc:1905] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:03:07.600270 29376 tablet_replica.cc:330] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e: stopping tablet replica
I20260929 03:03:07.600757 29376 raft_consensus.cc:2229] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:03:07.601342 29376 raft_consensus.cc:2258] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:03:07.601925 29374 tablet_replica.cc:330] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3: stopping tablet replica
I20260929 03:03:07.602344 29374 raft_consensus.cc:2229] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:03:07.602856 29374 raft_consensus.cc:2258] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:03:07.605391 29374 ts_tablet_manager.cc:1905] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:03:07.605543 29375 ts_tablet_manager.cc:1918] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:03:07.605783 29375 log.cc:1196] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-0/wal/wals/74ea0dbe38e143eab12eadac3172ac45
I20260929 03:03:07.606406 29375 ts_tablet_manager.cc:1939] T 74ea0dbe38e143eab12eadac3172ac45 P 0bdab4cad8c74ccf9b9eacc723f5d94b: Deleting consensus metadata
I20260929 03:03:07.607118 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.591284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.8.208.65:44729)
0929 03:03:07.591452 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:07.591461 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:07.597717 (+ 6256us) server_negotiation.cc:407] Connection header received
0929 03:03:07.598149 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:07.598156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:07.598254 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:07.598330 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:07.599041 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.600753 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.602790 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.603508 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.603621 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.605307 (+ 1686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:07.605326 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:07.605343 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:07.606182 (+ 839us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:07.606253 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:07.606280 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:07.606361 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:07.606631 (+ 270us) server_negotiation.cc:299] Negotiation successful
0929 03:03:07.606751 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 03:03:07.608476 29066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:07.608824 28869 catalog_manager.cc:4841] TS 0bdab4cad8c74ccf9b9eacc723f5d94b (127.8.208.65:44729): tablet 74ea0dbe38e143eab12eadac3172ac45 (table test-table [id=0a84563f60a147ef884c9c9d097b0475]) successfully deleted
I20260929 03:03:07.609591 29376 ts_tablet_manager.cc:1905] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:03:07.611176 29374 ts_tablet_manager.cc:1918] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:03:07.611336 29374 log.cc:1196] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-1/wal/wals/74ea0dbe38e143eab12eadac3172ac45
I20260929 03:03:07.611925 29374 ts_tablet_manager.cc:1939] T 74ea0dbe38e143eab12eadac3172ac45 P b5c81483a1a14f1793f1ae0247374fa3: Deleting consensus metadata
I20260929 03:03:07.614277 28872 catalog_manager.cc:4841] TS b5c81483a1a14f1793f1ae0247374fa3 (127.8.208.66:33827): tablet 74ea0dbe38e143eab12eadac3172ac45 (table test-table [id=0a84563f60a147ef884c9c9d097b0475]) successfully deleted
I20260929 03:03:07.617411 29376 ts_tablet_manager.cc:1918] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:03:07.617570 29376 log.cc:1196] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e: Deleting WAL directory at /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790650672582614-9025-0/minicluster-data/ts-2/wal/wals/74ea0dbe38e143eab12eadac3172ac45
I20260929 03:03:07.618013 29376 ts_tablet_manager.cc:1939] T 74ea0dbe38e143eab12eadac3172ac45 P 4083bcc2da714c8f862a7ac5a3c64e8e: Deleting consensus metadata
I20260929 03:03:07.619395 28869 catalog_manager.cc:4841] TS 4083bcc2da714c8f862a7ac5a3c64e8e (127.8.208.67:40951): tablet 74ea0dbe38e143eab12eadac3172ac45 (table test-table [id=0a84563f60a147ef884c9c9d097b0475]) successfully deleted
I20260929 03:03:07.709640 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 28944
I20260929 03:03:07.727211 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 29075
I20260929 03:03:07.735356 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.728098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45916 (local address 127.8.208.66:33827)
0929 03:03:07.728274 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:07.728280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:07.728301 (+ 21us) server_negotiation.cc:407] Connection header received
0929 03:03:07.728412 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:07.728422 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:07.728508 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:07.728627 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:07.729052 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.729861 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.731138 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.731560 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.731696 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.732719 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:07.732727 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:07.732741 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:07.733425 (+ 684us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:07.733517 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:07.733544 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:07.733678 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:07.733801 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 03:03:07.733901 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 03:03:07.735973 29197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:07.830446 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 29075
I20260929 03:03:07.849095 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 29205
I20260929 03:03:07.856956 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:07.849998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33628 (local address 127.8.208.67:40951)
0929 03:03:07.850205 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:07.850218 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:07.850296 (+ 78us) server_negotiation.cc:407] Connection header received
0929 03:03:07.850529 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:07.850534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:07.850633 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:07.850779 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:07.851237 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.852030 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.853528 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:07.853943 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:07.854095 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:07.855407 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:07.855426 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:07.855450 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:07.856109 (+ 659us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:07.856160 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:07.856182 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:07.856328 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:07.856492 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 03:03:07.856625 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 03:03:07.857620 29327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:07.949369 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 29205
I20260929 03:03:07.966460 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 28854
I20260929 03:03:07.967551 28915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:08.076151 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 28854
2026-09-29T03:03:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2773 ms)
[----------] 6 tests from EncryptionPolicyTest (21969 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:08 dist-test-slave-m1rv krb5kdc[29384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:08 dist-test-slave-m1rv krb5kdc[29384](info): set up 2 sockets
Sep 29 03:03:08 dist-test-slave-m1rv krb5kdc[29384](info): commencing operation
krb5kdc: starting...
W20260929 03:03:09.193819 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.005s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:09 dist-test-slave-m1rv krb5kdc[29384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:03:09.440778 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42417
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:09.775799 29397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:09.776135 29397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:09.776240 29397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:09.783576 29397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:09.783694 29397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:09.783746 29397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:09.783819 29397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:09.783885 29397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:09.793812 29397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42417
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:09.795084 29397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:09.796370 29397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:09.808789 29405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:09.808755 29402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:09.808751 29403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:09.809549 29397 server_base.cc:1034] running on GCE node
I20260929 03:03:09.810423 29397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:09.811825 29397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:09.813091 29397 hybrid_clock.cc:648] HybridClock initialized: now 1790650989813025 us; error 48 us; skew 500 ppm
I20260929 03:03:09.813508 29397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:09 dist-test-slave-m1rv krb5kdc[29384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:09.823853 29397 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:03:09.825556 29397 webserver.cc:460] Webserver started at http://127.0.0.1:43555/ using document root <none> and password file <none>
I20260929 03:03:09.826279 29397 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:09.826426 29397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:09.826723 29397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:09.829764 29397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "67dc2d1580be4850bed8afcff7b4b334"
format_stamp: "Formatted at 2026-09-29 03:03:09 on dist-test-slave-m1rv"
server_key: "8e53c4c918ad9b3fe00faa277f673cd4"
server_key_iv: "20156b18422c0538643404f28e06d520"
server_key_version: "encryptionkey@0"
I20260929 03:03:09.830531 29397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "67dc2d1580be4850bed8afcff7b4b334"
format_stamp: "Formatted at 2026-09-29 03:03:09 on dist-test-slave-m1rv"
server_key: "8e53c4c918ad9b3fe00faa277f673cd4"
server_key_iv: "20156b18422c0538643404f28e06d520"
server_key_version: "encryptionkey@0"
I20260929 03:03:09.835623 29397 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 03:03:09.839013 29412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:09.840240 29397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:03:09.840425 29397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "67dc2d1580be4850bed8afcff7b4b334"
format_stamp: "Formatted at 2026-09-29 03:03:09 on dist-test-slave-m1rv"
server_key: "8e53c4c918ad9b3fe00faa277f673cd4"
server_key_iv: "20156b18422c0538643404f28e06d520"
server_key_version: "encryptionkey@0"
I20260929 03:03:09.840619 29397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:09.874888 29397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:09.880827 29397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:09.881130 29397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:09.899794 29397 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42417
I20260929 03:03:09.899832 29463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42417 every 8 connection(s)
I20260929 03:03:09.901229 29397 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:09.904762 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 29397
I20260929 03:03:09.904894 29464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:09.905019 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:09.905716 9025 external_mini_cluster.cc:1428] Setting key a479eee33287b115ca25800d554d16fe
I20260929 03:03:09.916915 29464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334: Bootstrap starting.
I20260929 03:03:09.919847 29464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334: Neither blocks nor log segments found. Creating new log.
Sep 29 03:03:09 dist-test-slave-m1rv krb5kdc[29384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 03:03:09.920820 29464 log.cc:824] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:09.923648 29464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334: No bootstrap required, opened a new log
I20260929 03:03:09.928596 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:09.907626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44479 (local address 127.0.0.1:42417)
0929 03:03:09.908435 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:09.908448 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:09.908486 (+ 38us) server_negotiation.cc:407] Connection header received
0929 03:03:09.910988 (+ 2502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:09.911014 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:09.911421 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:09.911822 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:09.913284 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:09.914574 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:09.915901 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:09.916641 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:09.916836 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:09.921039 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:09.921051 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:09.921075 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:09.921109 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:09.924284 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:09.925106 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:09.925112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:09.925118 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:09.925361 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:09.925844 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:09.925848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:09.925850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:09.926395 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:09.926729 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:09.927246 (+ 517us) server_negotiation.cc:299] Negotiation successful
0929 03:03:09.927589 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":353,"threads_started":1}
I20260929 03:03:09.932181 29464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67dc2d1580be4850bed8afcff7b4b334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42417 } }
I20260929 03:03:09.932561 29464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:09.932689 29464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67dc2d1580be4850bed8afcff7b4b334, State: Initialized, Role: FOLLOWER
I20260929 03:03:09.932888 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:09.933359 29464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67dc2d1580be4850bed8afcff7b4b334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42417 } }
I20260929 03:03:09.933655 29464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:09.933789 29464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:09.933946 29464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:09.937247 29464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67dc2d1580be4850bed8afcff7b4b334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42417 } }
I20260929 03:03:09.937840 29464 leader_election.cc:302] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67dc2d1580be4850bed8afcff7b4b334; no voters:
I20260929 03:03:09.938542 29464 leader_election.cc:288] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:09.938691 29473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:09.941431 29464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:09.945775 29473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [term 1 LEADER]: Becoming Leader. State: Replica: 67dc2d1580be4850bed8afcff7b4b334, State: Running, Role: LEADER
I20260929 03:03:09.946514 29473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67dc2d1580be4850bed8afcff7b4b334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42417 } }
I20260929 03:03:09.949972 29475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67dc2d1580be4850bed8afcff7b4b334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67dc2d1580be4850bed8afcff7b4b334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42417 } } }
I20260929 03:03:09.950233 29474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67dc2d1580be4850bed8afcff7b4b334. Latest consensus state: current_term: 1 leader_uuid: "67dc2d1580be4850bed8afcff7b4b334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67dc2d1580be4850bed8afcff7b4b334" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42417 } } }
I20260929 03:03:09.950479 29475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:09.950563 29474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:09.952636 29481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:09.959677 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:09.938624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33425 (local address 127.0.0.1:42417)
0929 03:03:09.938807 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:09.938814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:09.940697 (+ 1883us) server_negotiation.cc:407] Connection header received
0929 03:03:09.940897 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:09.940903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:09.941001 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:09.941085 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:09.942738 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:09.943847 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:09.945495 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:09.946064 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:09.946208 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:09.949320 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:09.949332 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:09.949336 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:09.949374 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:09.954494 (+ 5120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:09.955825 (+ 1331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:09.955834 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:09.955840 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:09.956035 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:09.956725 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:09.956733 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:09.956739 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:09.957793 (+ 1054us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:09.958150 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:09.959124 (+ 974us) server_negotiation.cc:299] Negotiation successful
0929 03:03:09.959312 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":60}
I20260929 03:03:09.960693 29481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:09.969631 29481 catalog_manager.cc:1347] Generated new cluster ID: 410c7e8b75f44104bcfd80b6ed9c4061
I20260929 03:03:09.969738 29481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:09.979440 29481 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:09.981249 29481 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:09.989230 29481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67dc2d1580be4850bed8afcff7b4b334: Generated new TSK 0
I20260929 03:03:09.990170 29481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:09.999661 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 29397
I20260929 03:03:10.008328 29459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:10.101568 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 29397
W20260929 03:03:10.119078 29382 connection.cc:535] client connection to 127.0.0.1:42417 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42417): Connection reset by peer (error 104)
2026-09-29T03:03:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2021 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:10 dist-test-slave-m1rv krb5kdc[29496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:10 dist-test-slave-m1rv krb5kdc[29496](info): set up 2 sockets
Sep 29 03:03:10 dist-test-slave-m1rv krb5kdc[29496](info): commencing operation
krb5kdc: starting...
W20260929 03:03:11.213716 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:11 dist-test-slave-m1rv krb5kdc[29496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:03:11.464825 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44507
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:11.794212 29509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:11.794497 29509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:11.794589 29509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:11.801697 29509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:11.801800 29509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:11.801846 29509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:11.801906 29509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:11.801946 29509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:11.811601 29509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44507
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:11.812770 29509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:11.813988 29509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:11.825577 29515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:11.825670 29514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:11.825666 29517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:11.826362 29509 server_base.cc:1034] running on GCE node
I20260929 03:03:11.827104 29509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:11.828477 29509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:11.829701 29509 hybrid_clock.cc:648] HybridClock initialized: now 1790650991829653 us; error 46 us; skew 500 ppm
I20260929 03:03:11.830054 29509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:11 dist-test-slave-m1rv krb5kdc[29496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650991, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:11.833065 29509 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:03:11.834508 29509 webserver.cc:460] Webserver started at http://127.0.0.1:42549/ using document root <none> and password file <none>
I20260929 03:03:11.835184 29509 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:11.835336 29509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:11.835638 29509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:11.838690 29509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "ea6e458acf7b4e3184744cbbf276576c"
format_stamp: "Formatted at 2026-09-29 03:03:11 on dist-test-slave-m1rv"
server_key: "961d5930b1d569200cd46b6ac724a1e9"
server_key_iv: "417b74ee6f57515e670cf2532628b4ae"
server_key_version: "encryptionkey@0"
I20260929 03:03:11.839421 29509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "ea6e458acf7b4e3184744cbbf276576c"
format_stamp: "Formatted at 2026-09-29 03:03:11 on dist-test-slave-m1rv"
server_key: "961d5930b1d569200cd46b6ac724a1e9"
server_key_iv: "417b74ee6f57515e670cf2532628b4ae"
server_key_version: "encryptionkey@0"
I20260929 03:03:11.844329 29509 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 03:03:11.847754 29524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:11.849251 29509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:03:11.849434 29509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "ea6e458acf7b4e3184744cbbf276576c"
format_stamp: "Formatted at 2026-09-29 03:03:11 on dist-test-slave-m1rv"
server_key: "961d5930b1d569200cd46b6ac724a1e9"
server_key_iv: "417b74ee6f57515e670cf2532628b4ae"
server_key_version: "encryptionkey@0"
I20260929 03:03:11.849645 29509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:11.884409 29509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:11.888545 29509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:11.888893 29509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:11.907387 29509 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44507
I20260929 03:03:11.907418 29575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44507 every 8 connection(s)
I20260929 03:03:11.908809 29509 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:11.910198 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 29509
I20260929 03:03:11.910367 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:11.910948 9025 external_mini_cluster.cc:1428] Setting key bc37731a9bff430a26fe4140ed0e8bc3
I20260929 03:03:11.913719 29576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:11.923851 29579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:11.912870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49321 (local address 127.0.0.1:44507)
0929 03:03:11.913715 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:11.913728 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:11.913790 (+ 62us) server_negotiation.cc:407] Connection header received
0929 03:03:11.915418 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:11.915439 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:11.915820 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:11.916234 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:11.916787 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:11.918159 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:11.919481 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:11.920140 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:11.920334 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:11.921624 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:11.921632 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:11.921648 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:11.922231 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:11.922306 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:11.922333 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:11.922501 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:11.922784 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 03:03:11.922942 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":392,"threads_started":1}
I20260929 03:03:11.924398 29576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c: Bootstrap starting.
I20260929 03:03:11.927631 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:11.927651 29576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:11.928601 29576 log.cc:824] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:11.931267 29576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c: No bootstrap required, opened a new log
I20260929 03:03:11.939076 29579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:11.931953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37873 (local address 127.0.0.1:44507)
0929 03:03:11.932186 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:11.932193 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:11.932525 (+ 332us) server_negotiation.cc:407] Connection header received
0929 03:03:11.932754 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:11.932757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:11.932810 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:11.932881 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:11.933472 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:11.934541 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:11.936412 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:11.936897 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:11.937043 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:11.938266 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:11.938270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:11.938274 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:11.938289 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:11.938346 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:11.938361 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:11.938445 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:11.938673 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 03:03:11.938717 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 03:03:11.941260 29576 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6e458acf7b4e3184744cbbf276576c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44507 } }
I20260929 03:03:11.941642 29576 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:11.941749 29576 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6e458acf7b4e3184744cbbf276576c, State: Initialized, Role: FOLLOWER
I20260929 03:03:11.942365 29576 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6e458acf7b4e3184744cbbf276576c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44507 } }
I20260929 03:03:11.942605 29576 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:11.942764 29576 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:11.942936 29576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:11.945206 29576 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6e458acf7b4e3184744cbbf276576c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44507 } }
I20260929 03:03:11.945703 29576 leader_election.cc:302] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea6e458acf7b4e3184744cbbf276576c; no voters:
I20260929 03:03:11.946300 29576 leader_election.cc:288] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:11.946453 29586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:11.947151 29586 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [term 1 LEADER]: Becoming Leader. State: Replica: ea6e458acf7b4e3184744cbbf276576c, State: Running, Role: LEADER
I20260929 03:03:11.947695 29586 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6e458acf7b4e3184744cbbf276576c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44507 } }
I20260929 03:03:11.948387 29576 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:11.950273 29588 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea6e458acf7b4e3184744cbbf276576c. Latest consensus state: current_term: 1 leader_uuid: "ea6e458acf7b4e3184744cbbf276576c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6e458acf7b4e3184744cbbf276576c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44507 } } }
I20260929 03:03:11.950664 29588 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:11.955353 29587 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea6e458acf7b4e3184744cbbf276576c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6e458acf7b4e3184744cbbf276576c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44507 } } }
I20260929 03:03:11.955717 29587 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:11.957305 29592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:11.961795 29592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:11.970275 29592 catalog_manager.cc:1347] Generated new cluster ID: 673a67422bfc4b7ead410edef6b05217
I20260929 03:03:11.970392 29592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:11.981076 29592 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:11.982316 29592 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:11.988297 29592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea6e458acf7b4e3184744cbbf276576c: Generated new TSK 0
I20260929 03:03:11.989089 29592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:12.001565 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 29509
W20260929 03:03:12.002797 29571 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49321
W20260929 03:03:12.003574 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 29509: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 03:03:12.003760 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 29509
W20260929 03:03:12.020414 29494 connection.cc:535] client connection to 127.0.0.1:44507 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44507): Connection reset by peer (error 104)
2026-09-29T03:03:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1900 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:12 dist-test-slave-m1rv krb5kdc[29607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:12 dist-test-slave-m1rv krb5kdc[29607](info): set up 2 sockets
Sep 29 03:03:12 dist-test-slave-m1rv krb5kdc[29607](info): commencing operation
krb5kdc: starting...
W20260929 03:03:13.201668 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:13 dist-test-slave-m1rv krb5kdc[29607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:03:13.453344 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37467
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:13.777393 29620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:13.777688 29620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:13.777781 29620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:13.785013 29620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:13.785127 29620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:13.785228 29620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:13.785310 29620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:13.785355 29620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:13.795135 29620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37467
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:13.796391 29620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:13.797685 29620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:13.809669 29625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:13.809630 29628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:13.809602 29626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:13.810570 29620 server_base.cc:1034] running on GCE node
I20260929 03:03:13.811398 29620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:13.812826 29620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:13.814141 29620 hybrid_clock.cc:648] HybridClock initialized: now 1790650993814087 us; error 66 us; skew 500 ppm
I20260929 03:03:13.814541 29620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:13 dist-test-slave-m1rv krb5kdc[29607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650993, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:13.817837 29620 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:03:13.819311 29620 webserver.cc:460] Webserver started at http://127.0.0.1:43587/ using document root <none> and password file <none>
I20260929 03:03:13.820030 29620 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:13.820178 29620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:13.820483 29620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:13.823547 29620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "59ff13d04e404c1792c95ceca16d9993"
format_stamp: "Formatted at 2026-09-29 03:03:13 on dist-test-slave-m1rv"
server_key: "b5dd58a905668b4fd037a8d4fd0159db"
server_key_iv: "ac9911a6035fd8ef230df89edd4c1748"
server_key_version: "encryptionkey@0"
I20260929 03:03:13.824342 29620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "59ff13d04e404c1792c95ceca16d9993"
format_stamp: "Formatted at 2026-09-29 03:03:13 on dist-test-slave-m1rv"
server_key: "b5dd58a905668b4fd037a8d4fd0159db"
server_key_iv: "ac9911a6035fd8ef230df89edd4c1748"
server_key_version: "encryptionkey@0"
I20260929 03:03:13.829710 29620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260929 03:03:13.833340 29635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:13.834705 29620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:03:13.834913 29620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "59ff13d04e404c1792c95ceca16d9993"
format_stamp: "Formatted at 2026-09-29 03:03:13 on dist-test-slave-m1rv"
server_key: "b5dd58a905668b4fd037a8d4fd0159db"
server_key_iv: "ac9911a6035fd8ef230df89edd4c1748"
server_key_version: "encryptionkey@0"
I20260929 03:03:13.835145 29620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:13.840852 29620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:13.845561 29620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:13.845871 29620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:13.864506 29620 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37467
I20260929 03:03:13.864555 29686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37467 every 8 connection(s)
I20260929 03:03:13.866075 29620 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:13.866487 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 29620
I20260929 03:03:13.866676 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:13.867235 9025 external_mini_cluster.cc:1428] Setting key 9ff772832f4ca165fa1d82fed72b73f1
I20260929 03:03:13.870878 29687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:13.878623 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:13.869901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39553 (local address 127.0.0.1:37467)
0929 03:03:13.870650 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:13.870662 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:13.870702 (+ 40us) server_negotiation.cc:407] Connection header received
0929 03:03:13.873530 (+ 2828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:13.873564 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:13.874039 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:13.874558 (+ 519us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:13.876003 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:13.876011 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:13.876028 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:13.876045 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:13.876138 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:13.876173 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:13.876423 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:13.877058 (+ 635us) server_negotiation.cc:299] Negotiation successful
0929 03:03:13.877284 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":319,"threads_started":1}
I20260929 03:03:13.883332 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:13.886554 29687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993: Bootstrap starting.
I20260929 03:03:13.890273 29687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:13.890272 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:13.887829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34775 (local address 127.0.0.1:37467)
0929 03:03:13.888057 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:13.888068 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:13.888324 (+ 256us) server_negotiation.cc:407] Connection header received
0929 03:03:13.888461 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:13.888469 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:13.888541 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:13.888663 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:13.889553 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:13.889559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:13.889563 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:13.889580 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:13.889626 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:13.889645 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:13.889754 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:13.889864 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 03:03:13.889921 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 03:03:13.891613 29687 log.cc:824] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:13.894321 29687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993: No bootstrap required, opened a new log
I20260929 03:03:13.901928 29687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ff13d04e404c1792c95ceca16d9993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37467 } }
I20260929 03:03:13.902442 29687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:13.902565 29687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59ff13d04e404c1792c95ceca16d9993, State: Initialized, Role: FOLLOWER
I20260929 03:03:13.903218 29687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ff13d04e404c1792c95ceca16d9993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37467 } }
I20260929 03:03:13.903456 29687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:13.903622 29687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:13.903808 29687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:13.906080 29687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ff13d04e404c1792c95ceca16d9993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37467 } }
I20260929 03:03:13.906584 29687 leader_election.cc:302] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59ff13d04e404c1792c95ceca16d9993; no voters:
I20260929 03:03:13.907187 29687 leader_election.cc:288] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:13.907327 29697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:13.908041 29697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [term 1 LEADER]: Becoming Leader. State: Replica: 59ff13d04e404c1792c95ceca16d9993, State: Running, Role: LEADER
I20260929 03:03:13.908603 29697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ff13d04e404c1792c95ceca16d9993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37467 } }
I20260929 03:03:13.909464 29687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:13.911875 29698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59ff13d04e404c1792c95ceca16d9993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ff13d04e404c1792c95ceca16d9993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37467 } } }
I20260929 03:03:13.912142 29699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59ff13d04e404c1792c95ceca16d9993. Latest consensus state: current_term: 1 leader_uuid: "59ff13d04e404c1792c95ceca16d9993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ff13d04e404c1792c95ceca16d9993" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37467 } } }
I20260929 03:03:13.912545 29699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:13.913319 29698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:13.914456 29706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:13.918821 29706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:13.927330 29706 catalog_manager.cc:1347] Generated new cluster ID: 194e1d8575fe4fb8858fbea36951032e
I20260929 03:03:13.927459 29706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:13.936088 29706 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:13.936635 29706 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:13.943682 29706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59ff13d04e404c1792c95ceca16d9993: Generated new TSK 0
I20260929 03:03:13.944721 29706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:13.950690 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 29620
W20260929 03:03:13.951825 29682 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39553
W20260929 03:03:13.952513 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 29620: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 03:03:13.952656 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 29620
2026-09-29T03:03:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1948 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:14 dist-test-slave-m1rv krb5kdc[29718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:14 dist-test-slave-m1rv krb5kdc[29718](info): set up 2 sockets
Sep 29 03:03:14 dist-test-slave-m1rv krb5kdc[29718](info): commencing operation
krb5kdc: starting...
W20260929 03:03:15.053579 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:15 dist-test-slave-m1rv krb5kdc[29718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:03:15.305363 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38347
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:15.631557 29731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:15.631860 29731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:15.632043 29731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:15.639405 29731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:15.639518 29731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:15.639616 29731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:15.639701 29731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:15.639780 29731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:15.649583 29731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:15.650866 29731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:15.652213 29731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:15.665057 29737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:15.665112 29736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:15.665108 29739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:15.666030 29731 server_base.cc:1034] running on GCE node
I20260929 03:03:15.666734 29731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:15.668035 29731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:15.669286 29731 hybrid_clock.cc:648] HybridClock initialized: now 1790650995669246 us; error 52 us; skew 500 ppm
I20260929 03:03:15.669636 29731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:15 dist-test-slave-m1rv krb5kdc[29718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650995, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:15.679638 29731 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:03:15.681245 29731 webserver.cc:460] Webserver started at http://127.0.0.1:33937/ using document root <none> and password file <none>
I20260929 03:03:15.681965 29731 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:15.682111 29731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:15.682425 29731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:15.685623 29731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "abb30c03269249da85136e5129a19cf7"
format_stamp: "Formatted at 2026-09-29 03:03:15 on dist-test-slave-m1rv"
server_key: "fc0b5ebd0dafc0e3ec1594ba87e45eb0"
server_key_iv: "11ea94b49f6acba8781e993a4e0f25e3"
server_key_version: "encryptionkey@0"
I20260929 03:03:15.686378 29731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "abb30c03269249da85136e5129a19cf7"
format_stamp: "Formatted at 2026-09-29 03:03:15 on dist-test-slave-m1rv"
server_key: "fc0b5ebd0dafc0e3ec1594ba87e45eb0"
server_key_iv: "11ea94b49f6acba8781e993a4e0f25e3"
server_key_version: "encryptionkey@0"
I20260929 03:03:15.691360 29731 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 03:03:15.694748 29746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:15.696144 29731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:03:15.696331 29731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "abb30c03269249da85136e5129a19cf7"
format_stamp: "Formatted at 2026-09-29 03:03:15 on dist-test-slave-m1rv"
server_key: "fc0b5ebd0dafc0e3ec1594ba87e45eb0"
server_key_iv: "11ea94b49f6acba8781e993a4e0f25e3"
server_key_version: "encryptionkey@0"
I20260929 03:03:15.696533 29731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:15.714684 29731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:15.719297 29731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:15.719615 29731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:15.738191 29731 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38347
I20260929 03:03:15.738221 29797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38347 every 8 connection(s)
I20260929 03:03:15.739638 29731 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:15.740387 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 29731
I20260929 03:03:15.740586 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:15.741101 9025 external_mini_cluster.cc:1428] Setting key d62174972785eac9c63fbe90adce749a
I20260929 03:03:15.744051 29798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 03:03:15 dist-test-slave-m1rv krb5kdc[29718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 03:03:15.757386 29798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7: Bootstrap starting.
I20260929 03:03:15.760531 29798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:15.761638 29798 log.cc:824] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:15.764371 29801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:15.742994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37969 (local address 127.0.0.1:38347)
0929 03:03:15.743775 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:15.743786 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:15.743823 (+ 37us) server_negotiation.cc:407] Connection header received
0929 03:03:15.745500 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:15.745522 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:15.745888 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:15.746203 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:15.747995 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:15.749414 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:15.751275 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:15.751935 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:15.752065 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:15.757059 (+ 4994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:15.757066 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:15.757083 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:15.757123 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:15.760301 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:15.761063 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:15.761070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:15.761076 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:15.761305 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:15.761704 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:15.761709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:15.761712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:15.762255 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:15.762594 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:15.763018 (+ 424us) server_negotiation.cc:299] Negotiation successful
0929 03:03:15.763442 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":374,"threads_started":1}
I20260929 03:03:15.764642 29798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7: No bootstrap required, opened a new log
I20260929 03:03:15.768036 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:15.772639 29798 raft_consensus.cc:348] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb30c03269249da85136e5129a19cf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38347 } }
I20260929 03:03:15.773145 29798 raft_consensus.cc:374] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:15.773269 29798 raft_consensus.cc:729] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb30c03269249da85136e5129a19cf7, State: Initialized, Role: FOLLOWER
I20260929 03:03:15.773981 29798 consensus_queue.cc:260] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb30c03269249da85136e5129a19cf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38347 } }
I20260929 03:03:15.774300 29798 raft_consensus.cc:388] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:15.774456 29798 raft_consensus.cc:482] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:15.774641 29798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:15.777659 29798 raft_consensus.cc:504] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb30c03269249da85136e5129a19cf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38347 } }
I20260929 03:03:15.778259 29798 leader_election.cc:302] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abb30c03269249da85136e5129a19cf7; no voters:
I20260929 03:03:15.778950 29798 leader_election.cc:288] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:15.779026 29808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:15.779870 29808 raft_consensus.cc:686] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [term 1 LEADER]: Becoming Leader. State: Replica: abb30c03269249da85136e5129a19cf7, State: Running, Role: LEADER
I20260929 03:03:15.780515 29808 consensus_queue.cc:237] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb30c03269249da85136e5129a19cf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38347 } }
I20260929 03:03:15.781441 29798 sys_catalog.cc:564] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:15.789914 29810 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abb30c03269249da85136e5129a19cf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb30c03269249da85136e5129a19cf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38347 } } }
I20260929 03:03:15.790295 29810 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:15.790748 29809 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abb30c03269249da85136e5129a19cf7. Latest consensus state: current_term: 1 leader_uuid: "abb30c03269249da85136e5129a19cf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb30c03269249da85136e5129a19cf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38347 } } }
I20260929 03:03:15.791076 29809 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:15.791131 29824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:15.791129 29801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:15.772313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44863 (local address 127.0.0.1:38347)
0929 03:03:15.772545 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:15.772554 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:15.772922 (+ 368us) server_negotiation.cc:407] Connection header received
0929 03:03:15.773142 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:15.773146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:15.773283 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:15.773368 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:15.775363 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:15.776193 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:15.777996 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:15.778528 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:15.778630 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:15.781769 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:15.781776 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:15.781781 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:15.781826 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:15.787028 (+ 5202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:15.787856 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:15.787861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:15.787864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:15.788151 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:15.788687 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:15.788694 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:15.788700 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:15.789265 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:15.789595 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:15.789956 (+ 361us) server_negotiation.cc:299] Negotiation successful
0929 03:03:15.790273 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 03:03:15.795640 29824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:15.803759 29824 catalog_manager.cc:1347] Generated new cluster ID: f8cca988cdf74a8c97676fcc97ddcc4e
I20260929 03:03:15.803876 29824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:15.829936 29824 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:15.831278 29824 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:15.843546 29824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abb30c03269249da85136e5129a19cf7: Generated new TSK 0
I20260929 03:03:15.844697 29824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:15.853742 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 29731
I20260929 03:03:15.855506 29793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:15.952843 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 29731
2026-09-29T03:03:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2000 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:16 dist-test-slave-m1rv krb5kdc[29830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:16 dist-test-slave-m1rv krb5kdc[29830](info): set up 2 sockets
Sep 29 03:03:16 dist-test-slave-m1rv krb5kdc[29830](info): commencing operation
krb5kdc: starting...
W20260929 03:03:17.061650 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:17 dist-test-slave-m1rv krb5kdc[29830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:03:17.320915 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42641
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:17.650661 29843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:17.650977 29843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:17.651158 29843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:17.658576 29843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:17.658699 29843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:17.658819 29843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:17.658936 29843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:17.659011 29843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:17.670610 29843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:17.671979 29843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:17.673463 29843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:17.685874 29848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:17.685891 29851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:17.685945 29849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:17.686645 29843 server_base.cc:1034] running on GCE node
I20260929 03:03:17.687428 29843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:17.688840 29843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:17.690070 29843 hybrid_clock.cc:648] HybridClock initialized: now 1790650997690036 us; error 44 us; skew 500 ppm
I20260929 03:03:17.690430 29843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:17 dist-test-slave-m1rv krb5kdc[29830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650997, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:17.693571 29843 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:03:17.695134 29843 webserver.cc:460] Webserver started at http://127.0.0.1:44463/ using document root <none> and password file <none>
I20260929 03:03:17.695856 29843 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:17.696025 29843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:17.696336 29843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:17.699389 29843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "68555a229b2448bf9cf74b634049b4af"
format_stamp: "Formatted at 2026-09-29 03:03:17 on dist-test-slave-m1rv"
server_key: "d9b4db1697c748b9cc2fef9e9a62dcb5"
server_key_iv: "fc95f357383c8d7db1cb95b9fc46809a"
server_key_version: "encryptionkey@0"
I20260929 03:03:17.700145 29843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "68555a229b2448bf9cf74b634049b4af"
format_stamp: "Formatted at 2026-09-29 03:03:17 on dist-test-slave-m1rv"
server_key: "d9b4db1697c748b9cc2fef9e9a62dcb5"
server_key_iv: "fc95f357383c8d7db1cb95b9fc46809a"
server_key_version: "encryptionkey@0"
I20260929 03:03:17.705469 29843 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 03:03:17.708839 29858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:17.710289 29843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:03:17.710518 29843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "68555a229b2448bf9cf74b634049b4af"
format_stamp: "Formatted at 2026-09-29 03:03:17 on dist-test-slave-m1rv"
server_key: "d9b4db1697c748b9cc2fef9e9a62dcb5"
server_key_iv: "fc95f357383c8d7db1cb95b9fc46809a"
server_key_version: "encryptionkey@0"
I20260929 03:03:17.710743 29843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:17.730639 29843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:17.735435 29843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:17.735778 29843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:17.756506 29843 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42641
I20260929 03:03:17.756518 29909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42641 every 8 connection(s)
I20260929 03:03:17.758116 29843 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:17.761888 29910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:17.765481 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 29843
I20260929 03:03:17.765695 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:17.766204 9025 external_mini_cluster.cc:1428] Setting key f39ef13cbded6293e605c5b4b048f69f
I20260929 03:03:17.774777 29910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af: Bootstrap starting.
I20260929 03:03:17.778546 29910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:17.779713 29910 log.cc:824] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:17.780232 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:17.768057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51721 (local address 127.0.0.1:42641)
0929 03:03:17.768892 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:17.768905 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:17.768950 (+ 45us) server_negotiation.cc:407] Connection header received
0929 03:03:17.770680 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:17.770704 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:17.771212 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:17.771585 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:17.772258 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:17.773693 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:17.775533 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:17.776228 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:17.776375 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:17.777776 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:17.777783 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:17.777797 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:17.778330 (+ 533us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:17.778405 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:17.778435 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:17.778535 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:17.778880 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 03:03:17.779213 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":405,"threads_started":1}
I20260929 03:03:17.783156 29910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af: No bootstrap required, opened a new log
I20260929 03:03:17.784476 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:17.792007 29910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68555a229b2448bf9cf74b634049b4af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } }
I20260929 03:03:17.792412 29910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:17.792536 29910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68555a229b2448bf9cf74b634049b4af, State: Initialized, Role: FOLLOWER
I20260929 03:03:17.793220 29910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68555a229b2448bf9cf74b634049b4af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } }
I20260929 03:03:17.793534 29910 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:17.793692 29910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:17.793888 29910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:17.795560 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:17.788924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51019 (local address 127.0.0.1:42641)
0929 03:03:17.789121 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:17.789128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:17.789433 (+ 305us) server_negotiation.cc:407] Connection header received
0929 03:03:17.789599 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:17.789604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:17.789691 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:17.789810 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:17.790433 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:17.791532 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:17.792980 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:17.793589 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:17.793728 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:17.794933 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:17.794937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:17.794941 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:17.794958 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:17.794995 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:17.795013 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:17.795084 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:17.795222 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 03:03:17.795292 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 03:03:17.797315 29910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68555a229b2448bf9cf74b634049b4af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } }
I20260929 03:03:17.797839 29910 leader_election.cc:302] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68555a229b2448bf9cf74b634049b4af; no voters:
I20260929 03:03:17.798435 29910 leader_election.cc:288] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:17.798494 29920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:17.799301 29920 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [term 1 LEADER]: Becoming Leader. State: Replica: 68555a229b2448bf9cf74b634049b4af, State: Running, Role: LEADER
I20260929 03:03:17.799803 29920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68555a229b2448bf9cf74b634049b4af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } }
I20260929 03:03:17.800596 29910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:17.802922 29921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68555a229b2448bf9cf74b634049b4af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68555a229b2448bf9cf74b634049b4af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } } }
I20260929 03:03:17.802902 29922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68555a229b2448bf9cf74b634049b4af. Latest consensus state: current_term: 1 leader_uuid: "68555a229b2448bf9cf74b634049b4af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68555a229b2448bf9cf74b634049b4af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } } }
I20260929 03:03:17.803929 29921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:17.803947 29922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:17.805724 29929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:17.810942 29929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:17.820495 29929 catalog_manager.cc:1347] Generated new cluster ID: 3aa5767f213b4b7a82c0d483b17447b2
I20260929 03:03:17.820604 29929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:17.836763 29929 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:17.838169 29929 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:17.850533 29929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68555a229b2448bf9cf74b634049b4af: Generated new TSK 0
I20260929 03:03:17.851464 29929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:17.864578 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 29843
W20260929 03:03:17.865628 29905 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51721
W20260929 03:03:17.866348 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 29843: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 03:03:17.866505 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 29843
2026-09-29T03:03:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1917 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:17 dist-test-slave-m1rv krb5kdc[29941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:17 dist-test-slave-m1rv krb5kdc[29941](info): set up 2 sockets
Sep 29 03:03:17 dist-test-slave-m1rv krb5kdc[29941](info): commencing operation
krb5kdc: starting...
W20260929 03:03:18.982003 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:19 dist-test-slave-m1rv krb5kdc[29941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:03:19.246340 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36361
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:19.583328 29954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:19.583628 29954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:19.583793 29954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:19.591197 29954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:19.591321 29954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:19.591413 29954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:19.591521 29954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:19.591610 29954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:19.602092 29954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:19.603411 29954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:19.604733 29954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:19.616575 29959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:19.616621 29960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:19.616515 29962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:19.617874 29954 server_base.cc:1034] running on GCE node
I20260929 03:03:19.618662 29954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:19.620090 29954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:19.621306 29954 hybrid_clock.cc:648] HybridClock initialized: now 1790650999621249 us; error 56 us; skew 500 ppm
I20260929 03:03:19.621675 29954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:19 dist-test-slave-m1rv krb5kdc[29941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790650999, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:19.624737 29954 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:03:19.626245 29954 webserver.cc:460] Webserver started at http://127.0.0.1:46183/ using document root <none> and password file <none>
I20260929 03:03:19.626931 29954 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:19.627065 29954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:19.627380 29954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:19.630398 29954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "d1970a158a3f45baafb7903b3ff8dea1"
format_stamp: "Formatted at 2026-09-29 03:03:19 on dist-test-slave-m1rv"
server_key: "1dbe9d52b309c240acb27ac322d03fe7"
server_key_iv: "9704a6d6fe6404d889df95a656d4fc52"
server_key_version: "encryptionkey@0"
I20260929 03:03:19.631150 29954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "d1970a158a3f45baafb7903b3ff8dea1"
format_stamp: "Formatted at 2026-09-29 03:03:19 on dist-test-slave-m1rv"
server_key: "1dbe9d52b309c240acb27ac322d03fe7"
server_key_iv: "9704a6d6fe6404d889df95a656d4fc52"
server_key_version: "encryptionkey@0"
I20260929 03:03:19.636307 29954 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260929 03:03:19.639732 29969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:19.640983 29954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:03:19.641161 29954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "d1970a158a3f45baafb7903b3ff8dea1"
format_stamp: "Formatted at 2026-09-29 03:03:19 on dist-test-slave-m1rv"
server_key: "1dbe9d52b309c240acb27ac322d03fe7"
server_key_iv: "9704a6d6fe6404d889df95a656d4fc52"
server_key_version: "encryptionkey@0"
I20260929 03:03:19.641408 29954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:19.647315 29954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:19.651178 29954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:19.651481 29954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:19.670099 29954 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36361
I20260929 03:03:19.670192 30020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36361 every 8 connection(s)
I20260929 03:03:19.671792 29954 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:19.676163 30021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:19.681910 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 29954
I20260929 03:03:19.682109 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:19.682550 9025 external_mini_cluster.cc:1428] Setting key 3794b7789923e86a869850e908fa15cd
I20260929 03:03:19.687217 30021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1: Bootstrap starting.
I20260929 03:03:19.690304 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:19.684287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52825 (local address 127.0.0.1:36361)
0929 03:03:19.685142 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:19.685153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:19.685259 (+ 106us) server_negotiation.cc:407] Connection header received
0929 03:03:19.686925 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:19.686947 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:19.687283 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:19.687673 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:19.688735 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:19.688741 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:19.688753 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:19.688766 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:19.688828 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:19.688850 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:19.689022 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:19.689213 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 03:03:19.689446 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":411,"threads_started":1}
I20260929 03:03:19.690972 30021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:19.692209 30021 log.cc:824] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:19.694276 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:19.695179 30021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1: No bootstrap required, opened a new log
I20260929 03:03:19.702222 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:19.699625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50599 (local address 127.0.0.1:36361)
0929 03:03:19.699938 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:19.699945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:19.700180 (+ 235us) server_negotiation.cc:407] Connection header received
0929 03:03:19.700369 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:19.700374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:19.700427 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:19.700526 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:19.701520 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:19.701524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:19.701527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:19.701545 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:19.701581 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:19.701598 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:19.701701 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:19.701828 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 03:03:19.701876 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 03:03:19.704962 30021 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1970a158a3f45baafb7903b3ff8dea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36361 } }
I20260929 03:03:19.705436 30021 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:19.705556 30021 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1970a158a3f45baafb7903b3ff8dea1, State: Initialized, Role: FOLLOWER
I20260929 03:03:19.706127 30021 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1970a158a3f45baafb7903b3ff8dea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36361 } }
I20260929 03:03:19.706378 30021 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:19.706534 30021 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:19.706718 30021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:19.708897 30021 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1970a158a3f45baafb7903b3ff8dea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36361 } }
I20260929 03:03:19.709507 30021 leader_election.cc:302] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1970a158a3f45baafb7903b3ff8dea1; no voters:
I20260929 03:03:19.710282 30021 leader_election.cc:288] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:19.710455 30031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:19.711253 30031 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [term 1 LEADER]: Becoming Leader. State: Replica: d1970a158a3f45baafb7903b3ff8dea1, State: Running, Role: LEADER
I20260929 03:03:19.712018 30031 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1970a158a3f45baafb7903b3ff8dea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36361 } }
I20260929 03:03:19.712602 30021 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:19.715386 30032 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1970a158a3f45baafb7903b3ff8dea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1970a158a3f45baafb7903b3ff8dea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36361 } } }
I20260929 03:03:19.715400 30033 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1970a158a3f45baafb7903b3ff8dea1. Latest consensus state: current_term: 1 leader_uuid: "d1970a158a3f45baafb7903b3ff8dea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1970a158a3f45baafb7903b3ff8dea1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36361 } } }
I20260929 03:03:19.715898 30032 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:19.715893 30033 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:19.717636 30037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:19.724062 30037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:19.733656 30037 catalog_manager.cc:1347] Generated new cluster ID: 5bf5d646bde442cabba974b5d9017e4e
I20260929 03:03:19.733788 30037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:19.750603 30037 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:19.751116 30037 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:19.764057 30037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1970a158a3f45baafb7903b3ff8dea1: Generated new TSK 0
I20260929 03:03:19.765239 30037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:19.774879 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 29954
W20260929 03:03:19.775954 30016 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52825
W20260929 03:03:19.776643 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 29954: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 03:03:19.776785 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 29954
2026-09-29T03:03:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1905 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:19 dist-test-slave-m1rv krb5kdc[30052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:19 dist-test-slave-m1rv krb5kdc[30052](info): set up 2 sockets
Sep 29 03:03:19 dist-test-slave-m1rv krb5kdc[30052](info): commencing operation
krb5kdc: starting...
W20260929 03:03:20.885751 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.001s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:21 dist-test-slave-m1rv krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:03:21.143074 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:21.481071 30065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:21.481506 30065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:21.481698 30065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:21.488994 30065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:21.489164 30065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:21.489303 30065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:21.489457 30065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:21.489540 30065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:21.499786 30065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:21.501122 30065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:21.502506 30065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:21.514496 30070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:21.514640 30071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:21.514760 30073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:21.515933 30065 server_base.cc:1034] running on GCE node
I20260929 03:03:21.516708 30065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:21.518148 30065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:21.519349 30065 hybrid_clock.cc:648] HybridClock initialized: now 1790651001519312 us; error 30 us; skew 500 ppm
I20260929 03:03:21.519696 30065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:21 dist-test-slave-m1rv krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651001, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:21.529806 30065 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:03:21.531312 30065 webserver.cc:460] Webserver started at http://127.0.0.1:36133/ using document root <none> and password file <none>
I20260929 03:03:21.532006 30065 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:21.532109 30065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:21.532370 30065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:21.535332 30065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "e8ebf16869124004a68beba6924885c5"
format_stamp: "Formatted at 2026-09-29 03:03:21 on dist-test-slave-m1rv"
server_key: "8be2dd640f3c071030c7cead0042b2ab"
server_key_iv: "91333b15e4a17fcb209dd024ff4e9563"
server_key_version: "encryptionkey@0"
I20260929 03:03:21.536044 30065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "e8ebf16869124004a68beba6924885c5"
format_stamp: "Formatted at 2026-09-29 03:03:21 on dist-test-slave-m1rv"
server_key: "8be2dd640f3c071030c7cead0042b2ab"
server_key_iv: "91333b15e4a17fcb209dd024ff4e9563"
server_key_version: "encryptionkey@0"
I20260929 03:03:21.541224 30065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 03:03:21.544670 30080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:21.546321 30065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 03:03:21.546531 30065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "e8ebf16869124004a68beba6924885c5"
format_stamp: "Formatted at 2026-09-29 03:03:21 on dist-test-slave-m1rv"
server_key: "8be2dd640f3c071030c7cead0042b2ab"
server_key_iv: "91333b15e4a17fcb209dd024ff4e9563"
server_key_version: "encryptionkey@0"
I20260929 03:03:21.546753 30065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:21.566277 30065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:21.570886 30065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:21.571185 30065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:21.590493 30065 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35807
I20260929 03:03:21.590520 30131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35807 every 8 connection(s)
I20260929 03:03:21.591952 30065 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:21.595840 30132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:21.598439 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 30065
I20260929 03:03:21.598634 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:21.599171 9025 external_mini_cluster.cc:1428] Setting key a1c8f74e25162d3a1aede4872a689881
I20260929 03:03:21.608973 30132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5: Bootstrap starting.
I20260929 03:03:21.613062 30132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:21.614375 30132 log.cc:824] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5: Log is configured to *not* fsync() on all Append() calls
Sep 29 03:03:21 dist-test-slave-m1rv krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 03:03:21.617321 30132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5: No bootstrap required, opened a new log
I20260929 03:03:21.623250 30135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:21.601103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:54395 (local address 127.0.0.1:35807)
0929 03:03:21.602065 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:21.602078 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:21.602132 (+ 54us) server_negotiation.cc:407] Connection header received
0929 03:03:21.603919 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:21.603972 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:21.604415 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:21.604759 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:21.606592 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:21.607925 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:21.609785 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:21.610457 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:21.610605 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:21.615685 (+ 5080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:21.615694 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:21.615710 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:21.615746 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:21.618917 (+ 3171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:21.619785 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:21.619791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:21.619797 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:21.619964 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:21.620465 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:21.620468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:21.620470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:21.621000 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:21.621406 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:21.621889 (+ 483us) server_negotiation.cc:299] Negotiation successful
0929 03:03:21.622199 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":439,"threads_started":1}
I20260929 03:03:21.625670 30132 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ebf16869124004a68beba6924885c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35807 } }
I20260929 03:03:21.626116 30132 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:21.626226 30132 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ebf16869124004a68beba6924885c5, State: Initialized, Role: FOLLOWER
I20260929 03:03:21.626928 30132 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ebf16869124004a68beba6924885c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35807 } }
I20260929 03:03:21.627236 30132 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:21.627422 30132 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:21.627612 30132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:21.627655 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:21.630985 30132 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ebf16869124004a68beba6924885c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35807 } }
I20260929 03:03:21.631735 30132 leader_election.cc:302] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8ebf16869124004a68beba6924885c5; no voters:
I20260929 03:03:21.632495 30132 leader_election.cc:288] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:21.632643 30141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:21.633358 30141 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [term 1 LEADER]: Becoming Leader. State: Replica: e8ebf16869124004a68beba6924885c5, State: Running, Role: LEADER
I20260929 03:03:21.634001 30141 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ebf16869124004a68beba6924885c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35807 } }
I20260929 03:03:21.635990 30132 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:21.639173 30142 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8ebf16869124004a68beba6924885c5. Latest consensus state: current_term: 1 leader_uuid: "e8ebf16869124004a68beba6924885c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ebf16869124004a68beba6924885c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35807 } } }
I20260929 03:03:21.639586 30142 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:21.640218 30143 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8ebf16869124004a68beba6924885c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ebf16869124004a68beba6924885c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35807 } } }
I20260929 03:03:21.640566 30143 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:21.646191 30148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:21.654148 30148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:21.654556 30135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:21.634333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:49695 (local address 127.0.0.1:35807)
0929 03:03:21.634532 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:21.634538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:21.634560 (+ 22us) server_negotiation.cc:407] Connection header received
0929 03:03:21.634765 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:21.634770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:21.634854 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:21.634943 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:21.637380 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:21.638649 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:21.640742 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:21.641746 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:21.641880 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:21.646400 (+ 4520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:21.646406 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:21.646410 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:21.646443 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:21.650221 (+ 3778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:21.651329 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:21.651335 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:21.651339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:21.651570 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:21.652155 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:21.652159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:21.652162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:21.652594 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:21.652906 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:21.653876 (+ 970us) server_negotiation.cc:299] Negotiation successful
0929 03:03:21.654083 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 03:03:21.664351 30148 catalog_manager.cc:1347] Generated new cluster ID: cf3b93bc81c44bbc8654f333d4e19387
I20260929 03:03:21.664484 30148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:21.694208 30148 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:21.695521 30148 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:21.704219 30148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8ebf16869124004a68beba6924885c5: Generated new TSK 0
I20260929 03:03:21.705130 30148 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:21.721904 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30065
I20260929 03:03:21.723122 30127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:21.818964 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 30065
W20260929 03:03:21.836109 30050 connection.cc:535] client connection to 127.0.0.1:35807 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35807): Connection reset by peer (error 104)
2026-09-29T03:03:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2043 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:21 dist-test-slave-m1rv krb5kdc[30164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:21 dist-test-slave-m1rv krb5kdc[30164](info): set up 2 sockets
Sep 29 03:03:21 dist-test-slave-m1rv krb5kdc[30164](info): commencing operation
krb5kdc: starting...
W20260929 03:03:23.033627 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:23 dist-test-slave-m1rv krb5kdc[30164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:03:23.292801 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44677
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:23.622481 30177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:23.622767 30177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:23.622946 30177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:23.630067 30177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:23.630167 30177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:23.630263 30177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:23.630368 30177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:23.630434 30177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:23.640023 30177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:23.641299 30177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:23.642580 30177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:23.655802 30182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:23.655768 30185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:23.655858 30183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:23.656802 30177 server_base.cc:1034] running on GCE node
I20260929 03:03:23.657662 30177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:23.659060 30177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:23.660259 30177 hybrid_clock.cc:648] HybridClock initialized: now 1790651003660227 us; error 30 us; skew 500 ppm
I20260929 03:03:23.660609 30177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:23 dist-test-slave-m1rv krb5kdc[30164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:23.663749 30177 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:03:23.665293 30177 webserver.cc:460] Webserver started at http://127.0.0.1:44007/ using document root <none> and password file <none>
I20260929 03:03:23.665993 30177 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:23.666139 30177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:23.666437 30177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:23.669414 30177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "81e8730a05604d819d1f4f52cd0101be"
format_stamp: "Formatted at 2026-09-29 03:03:23 on dist-test-slave-m1rv"
server_key: "32f305bfaf3600cb6339fbbb08c23d91"
server_key_iv: "efc91f15135affaa2fe0f645ec4fe76c"
server_key_version: "encryptionkey@0"
I20260929 03:03:23.670167 30177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "81e8730a05604d819d1f4f52cd0101be"
format_stamp: "Formatted at 2026-09-29 03:03:23 on dist-test-slave-m1rv"
server_key: "32f305bfaf3600cb6339fbbb08c23d91"
server_key_iv: "efc91f15135affaa2fe0f645ec4fe76c"
server_key_version: "encryptionkey@0"
I20260929 03:03:23.675099 30177 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 03:03:23.678476 30192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:23.679816 30177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:03:23.679999 30177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "81e8730a05604d819d1f4f52cd0101be"
format_stamp: "Formatted at 2026-09-29 03:03:23 on dist-test-slave-m1rv"
server_key: "32f305bfaf3600cb6339fbbb08c23d91"
server_key_iv: "efc91f15135affaa2fe0f645ec4fe76c"
server_key_version: "encryptionkey@0"
I20260929 03:03:23.680190 30177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:23.706036 30177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:23.710322 30177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:23.710631 30177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:23.729862 30177 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44677
I20260929 03:03:23.729909 30243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44677 every 8 connection(s)
I20260929 03:03:23.731257 30177 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:23.735116 30244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:23.738536 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 30177
I20260929 03:03:23.738747 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:23.739200 9025 external_mini_cluster.cc:1428] Setting key 18d92f95851c2ae14913d19122e817bb
I20260929 03:03:23.747588 30244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be: Bootstrap starting.
I20260929 03:03:23.751513 30244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:23.752632 30244 log.cc:824] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:23.752825 30247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:23.740990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:33923 (local address 127.0.0.1:44677)
0929 03:03:23.741797 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:23.741808 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:23.741850 (+ 42us) server_negotiation.cc:407] Connection header received
0929 03:03:23.743477 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:23.743512 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:23.743938 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:23.744279 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:23.744933 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:23.746323 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:23.748197 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:23.748867 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:23.748998 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:23.750453 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:23.750460 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:23.750475 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:23.751035 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:23.751104 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:23.751130 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:23.751251 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:23.751579 (+ 328us) server_negotiation.cc:299] Negotiation successful
0929 03:03:23.751794 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":389,"threads_started":1}
I20260929 03:03:23.755719 30244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be: No bootstrap required, opened a new log
I20260929 03:03:23.756845 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:23.764339 30244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e8730a05604d819d1f4f52cd0101be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44677 } }
I20260929 03:03:23.764763 30244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:23.764875 30244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81e8730a05604d819d1f4f52cd0101be, State: Initialized, Role: FOLLOWER
I20260929 03:03:23.765620 30244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e8730a05604d819d1f4f52cd0101be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44677 } }
I20260929 03:03:23.765944 30244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:23.766100 30244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:23.766309 30244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:23.768164 30247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:23.761111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:33971 (local address 127.0.0.1:44677)
0929 03:03:23.761321 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:23.761327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:23.761806 (+ 479us) server_negotiation.cc:407] Connection header received
0929 03:03:23.761982 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:23.761988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:23.762072 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:23.762182 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:23.762652 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:23.763787 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:23.765342 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:23.766025 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:23.766172 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:23.767343 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:23.767348 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:23.767353 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:23.767376 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:23.767465 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:23.767487 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:23.767596 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:23.767762 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 03:03:23.767826 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 03:03:23.769081 30244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e8730a05604d819d1f4f52cd0101be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44677 } }
I20260929 03:03:23.769780 30244 leader_election.cc:302] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81e8730a05604d819d1f4f52cd0101be; no voters:
I20260929 03:03:23.770376 30244 leader_election.cc:288] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:23.770478 30254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:23.771238 30254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [term 1 LEADER]: Becoming Leader. State: Replica: 81e8730a05604d819d1f4f52cd0101be, State: Running, Role: LEADER
I20260929 03:03:23.771827 30254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e8730a05604d819d1f4f52cd0101be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44677 } }
I20260929 03:03:23.772650 30244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:23.775427 30255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81e8730a05604d819d1f4f52cd0101be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e8730a05604d819d1f4f52cd0101be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44677 } } }
I20260929 03:03:23.775804 30255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:23.777683 30262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:23.777658 30256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81e8730a05604d819d1f4f52cd0101be. Latest consensus state: current_term: 1 leader_uuid: "81e8730a05604d819d1f4f52cd0101be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e8730a05604d819d1f4f52cd0101be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44677 } } }
I20260929 03:03:23.778010 30256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:23.782866 30262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:23.793531 30262 catalog_manager.cc:1347] Generated new cluster ID: 0854e811564e46308193f419b3fbaf06
I20260929 03:03:23.793659 30262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:23.806238 30262 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:23.808023 30262 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:23.816843 30262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81e8730a05604d819d1f4f52cd0101be: Generated new TSK 0
I20260929 03:03:23.817920 30262 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:23.825343 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30177
W20260929 03:03:23.827384 30239 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.200:33923
W20260929 03:03:23.828107 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30177: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 03:03:23.828259 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 30177
W20260929 03:03:23.845253 30162 connection.cc:535] client connection to 127.0.0.1:44677 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44677): Connection reset by peer (error 104)
2026-09-29T03:03:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2008 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:23 dist-test-slave-m1rv krb5kdc[30275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:23 dist-test-slave-m1rv krb5kdc[30275](info): set up 2 sockets
Sep 29 03:03:23 dist-test-slave-m1rv krb5kdc[30275](info): commencing operation
krb5kdc: starting...
W20260929 03:03:24.950007 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:25 dist-test-slave-m1rv krb5kdc[30275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:03:25.201314 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:25.529645 30288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:25.529946 30288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:25.530112 30288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:25.537648 30288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:25.537755 30288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:25.537892 30288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:25.537982 30288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:25.538048 30288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:25.548480 30288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:25.549924 30288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:25.551271 30288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:25.563635 30294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:25.563604 30296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:25.564190 30293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:25.565301 30288 server_base.cc:1034] running on GCE node
I20260929 03:03:25.566211 30288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:25.567626 30288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:25.568868 30288 hybrid_clock.cc:648] HybridClock initialized: now 1790651005568830 us; error 52 us; skew 500 ppm
I20260929 03:03:25.569337 30288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:25 dist-test-slave-m1rv krb5kdc[30275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:25.572645 30288 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:03:25.574213 30288 webserver.cc:460] Webserver started at http://127.0.0.1:43029/ using document root <none> and password file <none>
I20260929 03:03:25.574895 30288 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:25.575055 30288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:25.575361 30288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:25.578383 30288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "3597f28e530f40068cf11d2608189298"
format_stamp: "Formatted at 2026-09-29 03:03:25 on dist-test-slave-m1rv"
server_key: "7a5cf382e4c5da355bb73c95a232c7d3"
server_key_iv: "8fd6b062805cfae7e8203270d172ed83"
server_key_version: "encryptionkey@0"
I20260929 03:03:25.579149 30288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "3597f28e530f40068cf11d2608189298"
format_stamp: "Formatted at 2026-09-29 03:03:25 on dist-test-slave-m1rv"
server_key: "7a5cf382e4c5da355bb73c95a232c7d3"
server_key_iv: "8fd6b062805cfae7e8203270d172ed83"
server_key_version: "encryptionkey@0"
I20260929 03:03:25.584285 30288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 03:03:25.587781 30303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:25.589098 30288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:03:25.589306 30288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "3597f28e530f40068cf11d2608189298"
format_stamp: "Formatted at 2026-09-29 03:03:25 on dist-test-slave-m1rv"
server_key: "7a5cf382e4c5da355bb73c95a232c7d3"
server_key_iv: "8fd6b062805cfae7e8203270d172ed83"
server_key_version: "encryptionkey@0"
I20260929 03:03:25.589506 30288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:25.595232 30288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:25.599452 30288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:25.599748 30288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:25.618093 30288 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44967
I20260929 03:03:25.618162 30354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44967 every 8 connection(s)
I20260929 03:03:25.619524 30288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:25.623308 30355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:25.625015 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 30288
I20260929 03:03:25.625227 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:25.625783 9025 external_mini_cluster.cc:1428] Setting key 5076d9a8ceeff01f719d16bf8818edf9
I20260929 03:03:25.635453 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:25.627941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:54441 (local address 127.0.0.1:44967)
0929 03:03:25.628906 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:25.628922 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:25.629377 (+ 455us) server_negotiation.cc:407] Connection header received
0929 03:03:25.631667 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:25.631694 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:25.632113 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:25.632567 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:25.633542 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:25.633550 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:25.633568 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:25.633588 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:25.633656 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:25.633680 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:25.633897 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:25.634033 (+ 136us) server_negotiation.cc:299] Negotiation successful
0929 03:03:25.634225 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":436,"threads_started":1}
I20260929 03:03:25.636688 30355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298: Bootstrap starting.
I20260929 03:03:25.639070 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:25.639889 30355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:25.640882 30355 log.cc:824] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:25.643859 30355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298: No bootstrap required, opened a new log
I20260929 03:03:25.645941 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:25.643493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:57479 (local address 127.0.0.1:44967)
0929 03:03:25.643666 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:25.643671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:25.644085 (+ 414us) server_negotiation.cc:407] Connection header received
0929 03:03:25.644212 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:25.644217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:25.644278 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:25.644412 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:25.645307 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:25.645312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:25.645316 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:25.645332 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:25.645383 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:25.645410 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:25.645532 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:25.645588 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 03:03:25.645643 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 03:03:25.651649 30355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3597f28e530f40068cf11d2608189298" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44967 } }
I20260929 03:03:25.652020 30355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:25.652140 30355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3597f28e530f40068cf11d2608189298, State: Initialized, Role: FOLLOWER
I20260929 03:03:25.652688 30355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3597f28e530f40068cf11d2608189298" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44967 } }
I20260929 03:03:25.652920 30355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:25.653066 30355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:25.653241 30355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:25.655428 30355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3597f28e530f40068cf11d2608189298" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44967 } }
I20260929 03:03:25.655901 30355 leader_election.cc:302] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3597f28e530f40068cf11d2608189298; no voters:
I20260929 03:03:25.656419 30355 leader_election.cc:288] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:25.656562 30365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:25.657260 30365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [term 1 LEADER]: Becoming Leader. State: Replica: 3597f28e530f40068cf11d2608189298, State: Running, Role: LEADER
I20260929 03:03:25.657805 30365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3597f28e530f40068cf11d2608189298" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44967 } }
I20260929 03:03:25.658492 30355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:25.660633 30367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3597f28e530f40068cf11d2608189298. Latest consensus state: current_term: 1 leader_uuid: "3597f28e530f40068cf11d2608189298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3597f28e530f40068cf11d2608189298" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44967 } } }
I20260929 03:03:25.661089 30367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:25.661661 30366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3597f28e530f40068cf11d2608189298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3597f28e530f40068cf11d2608189298" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44967 } } }
I20260929 03:03:25.661978 30366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:25.662554 30370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:25.668179 30370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:25.676005 30370 catalog_manager.cc:1347] Generated new cluster ID: 0ba49532cfb045448844427ba907024c
I20260929 03:03:25.676107 30370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:25.694655 30370 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:25.695192 30370 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:25.706651 30370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3597f28e530f40068cf11d2608189298: Generated new TSK 0
I20260929 03:03:25.707651 30370 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:25.714219 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30288
W20260929 03:03:25.715281 30350 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.200:54441
W20260929 03:03:25.715960 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30288: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 03:03:25.716122 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 30288
2026-09-29T03:03:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1887 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:25 dist-test-slave-m1rv krb5kdc[30386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:25 dist-test-slave-m1rv krb5kdc[30386](info): set up 2 sockets
Sep 29 03:03:25 dist-test-slave-m1rv krb5kdc[30386](info): commencing operation
krb5kdc: starting...
W20260929 03:03:26.809705 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:26 dist-test-slave-m1rv krb5kdc[30386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:03:27.056180 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:27.387030 30399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:27.387339 30399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:27.387509 30399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:27.394939 30399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:27.395071 30399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:27.395193 30399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:27.395303 30399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:27.395378 30399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:27.405086 30399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44359
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:27.406349 30399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:27.407640 30399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:27.419219 30404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:27.419293 30407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:27.419255 30405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:27.419903 30399 server_base.cc:1034] running on GCE node
I20260929 03:03:27.420744 30399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:27.422124 30399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:27.423323 30399 hybrid_clock.cc:648] HybridClock initialized: now 1790651007423265 us; error 52 us; skew 500 ppm
I20260929 03:03:27.423700 30399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:27 dist-test-slave-m1rv krb5kdc[30386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651007, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:27.432238 30399 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:03:27.433660 30399 webserver.cc:460] Webserver started at http://127.0.0.1:46297/ using document root <none> and password file <none>
I20260929 03:03:27.434362 30399 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:27.434490 30399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:27.434793 30399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:27.437844 30399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "087f0856b2c0492694f525a8ca075e43"
format_stamp: "Formatted at 2026-09-29 03:03:27 on dist-test-slave-m1rv"
server_key: "249538bce27f1ce5b316df971348f131"
server_key_iv: "cc1f0445a6f8a27f11495f9e49af3f8d"
server_key_version: "encryptionkey@0"
I20260929 03:03:27.438596 30399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "087f0856b2c0492694f525a8ca075e43"
format_stamp: "Formatted at 2026-09-29 03:03:27 on dist-test-slave-m1rv"
server_key: "249538bce27f1ce5b316df971348f131"
server_key_iv: "cc1f0445a6f8a27f11495f9e49af3f8d"
server_key_version: "encryptionkey@0"
I20260929 03:03:27.443434 30399 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260929 03:03:27.446627 30414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:27.447916 30399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:03:27.448110 30399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "087f0856b2c0492694f525a8ca075e43"
format_stamp: "Formatted at 2026-09-29 03:03:27 on dist-test-slave-m1rv"
server_key: "249538bce27f1ce5b316df971348f131"
server_key_iv: "cc1f0445a6f8a27f11495f9e49af3f8d"
server_key_version: "encryptionkey@0"
I20260929 03:03:27.448298 30399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:27.460541 30399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:27.464303 30399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:27.464597 30399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:27.482800 30399 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44359
I20260929 03:03:27.482832 30465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44359 every 8 connection(s)
I20260929 03:03:27.484200 30399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:27.487879 30466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:27.490120 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 30399
I20260929 03:03:27.490303 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:27.490762 9025 external_mini_cluster.cc:1428] Setting key 0ebf1296c85536cf993cf5bd3962db1b
I20260929 03:03:27.499727 30466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43: Bootstrap starting.
I20260929 03:03:27.503496 30466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:27.504827 30466 log.cc:824] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43: Log is configured to *not* fsync() on all Append() calls
Sep 29 03:03:27 dist-test-slave-m1rv krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 03:03:27.507841 30466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43: No bootstrap required, opened a new log
I20260929 03:03:27.514369 30469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:27.492620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:58555 (local address 127.0.0.1:44359)
0929 03:03:27.493483 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:27.493495 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:27.493537 (+ 42us) server_negotiation.cc:407] Connection header received
0929 03:03:27.495168 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:27.495189 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:27.495575 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:27.495891 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:27.497738 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:27.499018 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:27.500749 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:27.501486 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:27.501614 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:27.506763 (+ 5149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:27.506770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:27.506785 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:27.506818 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:27.510022 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:27.510805 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:27.510812 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:27.510818 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:27.510989 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:27.511458 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:27.511469 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:27.511472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:27.512004 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:27.512342 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:27.512859 (+ 517us) server_negotiation.cc:299] Negotiation successful
0929 03:03:27.513237 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":386,"threads_started":1}
I20260929 03:03:27.515975 30466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087f0856b2c0492694f525a8ca075e43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } }
I20260929 03:03:27.516443 30466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:27.516562 30466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 087f0856b2c0492694f525a8ca075e43, State: Initialized, Role: FOLLOWER
I20260929 03:03:27.517247 30466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087f0856b2c0492694f525a8ca075e43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } }
I20260929 03:03:27.517506 30466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:27.517653 30466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:27.517828 30466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:27.518514 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:27.520146 30466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087f0856b2c0492694f525a8ca075e43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } }
I20260929 03:03:27.520623 30466 leader_election.cc:302] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 087f0856b2c0492694f525a8ca075e43; no voters:
I20260929 03:03:27.521301 30466 leader_election.cc:288] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:27.521396 30472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:27.523860 30466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:27.528337 30472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [term 1 LEADER]: Becoming Leader. State: Replica: 087f0856b2c0492694f525a8ca075e43, State: Running, Role: LEADER
I20260929 03:03:27.529000 30472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087f0856b2c0492694f525a8ca075e43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } }
I20260929 03:03:27.532266 30475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "087f0856b2c0492694f525a8ca075e43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087f0856b2c0492694f525a8ca075e43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } } }
I20260929 03:03:27.532706 30475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:27.543675 30469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:27.526711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:50943 (local address 127.0.0.1:44359)
0929 03:03:27.527074 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:27.527082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:27.527407 (+ 325us) server_negotiation.cc:407] Connection header received
0929 03:03:27.527604 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:27.527609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:27.527679 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:27.527779 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:27.529210 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:27.530139 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:27.531704 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:27.532420 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:27.532534 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:27.536901 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:27.536911 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:27.536919 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:27.536968 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:27.540691 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:27.541586 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:27.541592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:27.541596 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:27.541709 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:27.542177 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:27.542182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:27.542185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:27.542468 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:27.542735 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:27.543123 (+ 388us) server_negotiation.cc:299] Negotiation successful
0929 03:03:27.543299 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260929 03:03:27.548944 30476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 087f0856b2c0492694f525a8ca075e43. Latest consensus state: current_term: 1 leader_uuid: "087f0856b2c0492694f525a8ca075e43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087f0856b2c0492694f525a8ca075e43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } } }
I20260929 03:03:27.549225 30482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:27.549263 30476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:27.555192 30482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:27.564477 30482 catalog_manager.cc:1347] Generated new cluster ID: 7c4f0480e70d4f2b9a8de92b6e15eadf
I20260929 03:03:27.564606 30482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:27.576643 30482 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:27.577950 30482 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:27.586634 30482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 087f0856b2c0492694f525a8ca075e43: Generated new TSK 0
I20260929 03:03:27.587374 30482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:27.594646 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30399
I20260929 03:03:27.596676 30461 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:27.685918 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 30399
W20260929 03:03:27.702409 30384 connection.cc:535] client connection to 127.0.0.1:44359 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44359): Connection reset by peer (error 104)
2026-09-29T03:03:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1969 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:27 dist-test-slave-m1rv krb5kdc[30499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:27 dist-test-slave-m1rv krb5kdc[30499](info): set up 2 sockets
Sep 29 03:03:27 dist-test-slave-m1rv krb5kdc[30499](info): commencing operation
krb5kdc: starting...
W20260929 03:03:28.789681 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:28 dist-test-slave-m1rv krb5kdc[30499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:03:29.048454 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:29.375583 30512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:29.375914 30512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:29.376091 30512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:29.383280 30512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:29.383420 30512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:29.383529 30512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:29.383630 30512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:29.383714 30512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:29.393347 30512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38981
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:29.394590 30512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:29.395875 30512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:29.407683 30520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:29.407717 30517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:29.407742 30518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:29.408195 30512 server_base.cc:1034] running on GCE node
I20260929 03:03:29.409390 30512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:29.410825 30512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:29.412055 30512 hybrid_clock.cc:648] HybridClock initialized: now 1790651009412015 us; error 42 us; skew 500 ppm
I20260929 03:03:29.412405 30512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:29 dist-test-slave-m1rv krb5kdc[30499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651009, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:29.415597 30512 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:03:29.417138 30512 webserver.cc:460] Webserver started at http://127.0.0.1:37895/ using document root <none> and password file <none>
I20260929 03:03:29.417975 30512 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:29.418081 30512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:29.418321 30512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:29.421250 30512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "1f15a813c6564561b3f0399dec9d530b"
format_stamp: "Formatted at 2026-09-29 03:03:29 on dist-test-slave-m1rv"
server_key: "0c861688dc9858a459e48914fc50594b"
server_key_iv: "92e14559b3cdb6d158843a4a099c5740"
server_key_version: "encryptionkey@0"
I20260929 03:03:29.421964 30512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "1f15a813c6564561b3f0399dec9d530b"
format_stamp: "Formatted at 2026-09-29 03:03:29 on dist-test-slave-m1rv"
server_key: "0c861688dc9858a459e48914fc50594b"
server_key_iv: "92e14559b3cdb6d158843a4a099c5740"
server_key_version: "encryptionkey@0"
I20260929 03:03:29.426939 30512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 03:03:29.430497 30527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:29.431951 30512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:03:29.432138 30512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "1f15a813c6564561b3f0399dec9d530b"
format_stamp: "Formatted at 2026-09-29 03:03:29 on dist-test-slave-m1rv"
server_key: "0c861688dc9858a459e48914fc50594b"
server_key_iv: "92e14559b3cdb6d158843a4a099c5740"
server_key_version: "encryptionkey@0"
I20260929 03:03:29.432332 30512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:29.470126 30512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:29.474180 30512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:29.474472 30512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:29.492882 30512 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38981
I20260929 03:03:29.492919 30578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38981 every 8 connection(s)
I20260929 03:03:29.494248 30512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:29.497938 30579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:29.503044 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 30512
I20260929 03:03:29.503223 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:29.503723 9025 external_mini_cluster.cc:1428] Setting key 26ac3ca2f6b2728e73cea33ed67a7361
I20260929 03:03:29.508870 30579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b: Bootstrap starting.
I20260929 03:03:29.512365 30579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:29.513581 30579 log.cc:824] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:29.516637 30579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b: No bootstrap required, opened a new log
I20260929 03:03:29.517150 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:29.505603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:36061 (local address 127.0.0.1:38981)
0929 03:03:29.506384 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:29.506396 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:29.506435 (+ 39us) server_negotiation.cc:407] Connection header received
0929 03:03:29.508147 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:29.508169 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:29.508566 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:29.508868 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:29.509518 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:29.510817 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:29.512658 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:29.513394 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:29.513540 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:29.514898 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:29.514908 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:29.514927 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:29.515465 (+ 538us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:29.515543 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:29.515567 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:29.515686 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:29.515952 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 03:03:29.516137 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":339,"threads_started":1}
I20260929 03:03:29.520960 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:29.526221 30579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f15a813c6564561b3f0399dec9d530b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } }
I20260929 03:03:29.526674 30579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:29.526787 30579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f15a813c6564561b3f0399dec9d530b, State: Initialized, Role: FOLLOWER
I20260929 03:03:29.527489 30579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f15a813c6564561b3f0399dec9d530b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } }
I20260929 03:03:29.527722 30579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:29.527886 30579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:29.528077 30579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:29.530382 30579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f15a813c6564561b3f0399dec9d530b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } }
I20260929 03:03:29.530884 30579 leader_election.cc:302] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f15a813c6564561b3f0399dec9d530b; no voters:
I20260929 03:03:29.531543 30579 leader_election.cc:288] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:29.531644 30589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:29.532868 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:29.526203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:40213 (local address 127.0.0.1:38981)
0929 03:03:29.526385 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:29.526391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:29.526804 (+ 413us) server_negotiation.cc:407] Connection header received
0929 03:03:29.527069 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:29.527074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:29.527151 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:29.527269 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:29.527729 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:29.528641 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:29.530013 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:29.530454 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:29.530585 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:29.531759 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:29.531769 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:29.531774 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:29.531794 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:29.531845 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:29.531862 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:29.531993 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:29.532269 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 03:03:29.532338 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 03:03:29.533018 30589 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [term 1 LEADER]: Becoming Leader. State: Replica: 1f15a813c6564561b3f0399dec9d530b, State: Running, Role: LEADER
I20260929 03:03:29.533634 30589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f15a813c6564561b3f0399dec9d530b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } }
I20260929 03:03:29.534571 30579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:29.535745 30590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f15a813c6564561b3f0399dec9d530b. Latest consensus state: current_term: 1 leader_uuid: "1f15a813c6564561b3f0399dec9d530b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f15a813c6564561b3f0399dec9d530b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } } }
I20260929 03:03:29.536211 30590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:29.537773 30591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f15a813c6564561b3f0399dec9d530b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f15a813c6564561b3f0399dec9d530b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38981 } } }
I20260929 03:03:29.538204 30591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:29.542526 30594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:29.547999 30594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:29.556577 30594 catalog_manager.cc:1347] Generated new cluster ID: 34c158815ed74ab2bac12bba899b64eb
I20260929 03:03:29.556700 30594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:29.568492 30594 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:29.570377 30594 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:29.587741 30594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f15a813c6564561b3f0399dec9d530b: Generated new TSK 0
I20260929 03:03:29.588768 30594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:29.604974 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30512
W20260929 03:03:29.606174 30574 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.200:36061
W20260929 03:03:29.606904 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30512: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 03:03:29.607060 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 30512
W20260929 03:03:29.623761 30497 connection.cc:535] client connection to 127.0.0.1:38981 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38981): Connection reset by peer (error 104)
2026-09-29T03:03:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1925 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:29 dist-test-slave-m1rv krb5kdc[30610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:29 dist-test-slave-m1rv krb5kdc[30610](info): set up 2 sockets
Sep 29 03:03:29 dist-test-slave-m1rv krb5kdc[30610](info): commencing operation
krb5kdc: starting...
W20260929 03:03:30.733788 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:30 dist-test-slave-m1rv krb5kdc[30610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:03:30.984745 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:31.319031 30623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:31.319365 30623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:31.319545 30623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:31.326764 30623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:31.326930 30623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:31.327039 30623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:31.327150 30623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:31.327239 30623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:31.336747 30623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43935
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:31.338052 30623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:31.339344 30623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:31.351253 30628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:31.351279 30631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:31.351200 30629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:31.352174 30623 server_base.cc:1034] running on GCE node
I20260929 03:03:31.352921 30623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:31.354383 30623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:31.355590 30623 hybrid_clock.cc:648] HybridClock initialized: now 1790651011355552 us; error 33 us; skew 500 ppm
I20260929 03:03:31.355928 30623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:31 dist-test-slave-m1rv krb5kdc[30610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651011, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:31.358983 30623 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:03:31.360334 30623 webserver.cc:460] Webserver started at http://127.0.0.1:42359/ using document root <none> and password file <none>
I20260929 03:03:31.361042 30623 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:31.361225 30623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:31.361536 30623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:31.364411 30623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "99cbf109ccdf4450a64f75a59d9329cb"
format_stamp: "Formatted at 2026-09-29 03:03:31 on dist-test-slave-m1rv"
server_key: "aa554b59c21687c3a65ed07d490dab31"
server_key_iv: "7e5dd0fbe2b9de8ad889906e9795f88c"
server_key_version: "encryptionkey@0"
I20260929 03:03:31.365216 30623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "99cbf109ccdf4450a64f75a59d9329cb"
format_stamp: "Formatted at 2026-09-29 03:03:31 on dist-test-slave-m1rv"
server_key: "aa554b59c21687c3a65ed07d490dab31"
server_key_iv: "7e5dd0fbe2b9de8ad889906e9795f88c"
server_key_version: "encryptionkey@0"
I20260929 03:03:31.370253 30623 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 03:03:31.373623 30638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:31.375094 30623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:03:31.375370 30623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "99cbf109ccdf4450a64f75a59d9329cb"
format_stamp: "Formatted at 2026-09-29 03:03:31 on dist-test-slave-m1rv"
server_key: "aa554b59c21687c3a65ed07d490dab31"
server_key_iv: "7e5dd0fbe2b9de8ad889906e9795f88c"
server_key_version: "encryptionkey@0"
I20260929 03:03:31.375586 30623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:31.381457 30623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:31.385336 30623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:31.385615 30623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:31.403396 30623 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43935
I20260929 03:03:31.403432 30689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43935 every 8 connection(s)
I20260929 03:03:31.404793 30623 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:31.407357 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 30623
I20260929 03:03:31.407531 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:31.408051 9025 external_mini_cluster.cc:1428] Setting key 807f6173e83cade98c74fa576327811b
I20260929 03:03:31.408996 30690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:31.415457 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:31.409819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:47121 (local address 127.0.0.1:43935)
0929 03:03:31.410593 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:31.410604 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:31.410642 (+ 38us) server_negotiation.cc:407] Connection header received
0929 03:03:31.412139 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:31.412158 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:31.412490 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:31.412807 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:31.413869 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:31.413875 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:31.413887 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:31.413900 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:31.414003 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:31.414023 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:31.414191 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:31.414314 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 03:03:31.414523 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":344,"threads_started":1}
I20260929 03:03:31.419039 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:31.422544 30690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb: Bootstrap starting.
I20260929 03:03:31.426206 30690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:31.426772 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:31.424092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.200:40889 (local address 127.0.0.1:43935)
0929 03:03:31.424304 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:31.424312 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:31.424775 (+ 463us) server_negotiation.cc:407] Connection header received
0929 03:03:31.424913 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:31.424918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:31.424974 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:31.425082 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:31.426008 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:31.426016 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:31.426023 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:31.426045 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:31.426100 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:31.426122 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:31.426257 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:31.426372 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 03:03:31.426438 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 03:03:31.427451 30690 log.cc:824] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:31.430109 30690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb: No bootstrap required, opened a new log
I20260929 03:03:31.437273 30690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99cbf109ccdf4450a64f75a59d9329cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } }
I20260929 03:03:31.437633 30690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:31.437743 30690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99cbf109ccdf4450a64f75a59d9329cb, State: Initialized, Role: FOLLOWER
I20260929 03:03:31.438345 30690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99cbf109ccdf4450a64f75a59d9329cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } }
I20260929 03:03:31.438578 30690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:31.438721 30690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:31.438890 30690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:31.441041 30690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99cbf109ccdf4450a64f75a59d9329cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } }
I20260929 03:03:31.441546 30690 leader_election.cc:302] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99cbf109ccdf4450a64f75a59d9329cb; no voters:
I20260929 03:03:31.442095 30690 leader_election.cc:288] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:31.442199 30700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:31.442899 30700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [term 1 LEADER]: Becoming Leader. State: Replica: 99cbf109ccdf4450a64f75a59d9329cb, State: Running, Role: LEADER
I20260929 03:03:31.443449 30700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99cbf109ccdf4450a64f75a59d9329cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } }
I20260929 03:03:31.444087 30690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:31.446334 30702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99cbf109ccdf4450a64f75a59d9329cb. Latest consensus state: current_term: 1 leader_uuid: "99cbf109ccdf4450a64f75a59d9329cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99cbf109ccdf4450a64f75a59d9329cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } } }
I20260929 03:03:31.446434 30701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99cbf109ccdf4450a64f75a59d9329cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99cbf109ccdf4450a64f75a59d9329cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43935 } } }
I20260929 03:03:31.446746 30702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:31.446802 30701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:31.448704 30706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:31.454870 30706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:31.464816 30706 catalog_manager.cc:1347] Generated new cluster ID: f5b7783a82824494a11e48998d7de4e7
I20260929 03:03:31.464946 30706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:31.477690 30706 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:31.478184 30706 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:31.494251 30706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99cbf109ccdf4450a64f75a59d9329cb: Generated new TSK 0
I20260929 03:03:31.495290 30706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:31.511675 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30623
W20260929 03:03:31.512770 30685 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.200:47121
W20260929 03:03:31.513540 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30623: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 03:03:31.513684 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 30623
2026-09-29T03:03:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1902 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:31 dist-test-slave-m1rv krb5kdc[30721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:31 dist-test-slave-m1rv krb5kdc[30721](info): set up 2 sockets
Sep 29 03:03:31 dist-test-slave-m1rv krb5kdc[30721](info): commencing operation
krb5kdc: starting...
W20260929 03:03:32.637717 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:32 dist-test-slave-m1rv krb5kdc[30721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:03:32.892010 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:36349
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:42895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:36349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:33.229871 30734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:33.230150 30734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:33.230245 30734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:33.237407 30734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:33.237519 30734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:33.237563 30734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:33.237615 30734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:33.237661 30734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:33.247359 30734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:42895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:36349
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:36349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:33.248581 30734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:33.249835 30734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:33.261533 30742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:33.261560 30739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:33.261706 30740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:33.262547 30734 server_base.cc:1034] running on GCE node
I20260929 03:03:33.263284 30734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:33.264696 30734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:33.265961 30734 hybrid_clock.cc:648] HybridClock initialized: now 1790651013265887 us; error 30 us; skew 500 ppm
I20260929 03:03:33.266273 30734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:33 dist-test-slave-m1rv krb5kdc[30721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651013, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:33.269357 30734 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:03:33.270723 30734 webserver.cc:460] Webserver started at http://127.8.208.126:33905/ using document root <none> and password file <none>
I20260929 03:03:33.271330 30734 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:33.271437 30734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:33.271682 30734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:33.274612 30734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "b49d3396b72746beb3ee786594511a36"
format_stamp: "Formatted at 2026-09-29 03:03:33 on dist-test-slave-m1rv"
server_key: "ade054ee0da48e4c07fea31f4e3aeb96"
server_key_iv: "7cf045137f5749bb73c293886b8b6a0d"
server_key_version: "encryptionkey@0"
I20260929 03:03:33.275343 30734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "b49d3396b72746beb3ee786594511a36"
format_stamp: "Formatted at 2026-09-29 03:03:33 on dist-test-slave-m1rv"
server_key: "ade054ee0da48e4c07fea31f4e3aeb96"
server_key_iv: "7cf045137f5749bb73c293886b8b6a0d"
server_key_version: "encryptionkey@0"
I20260929 03:03:33.280505 30734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 03:03:33.283936 30749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:33.285341 30734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:03:33.285522 30734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "b49d3396b72746beb3ee786594511a36"
format_stamp: "Formatted at 2026-09-29 03:03:33 on dist-test-slave-m1rv"
server_key: "ade054ee0da48e4c07fea31f4e3aeb96"
server_key_iv: "7cf045137f5749bb73c293886b8b6a0d"
server_key_version: "encryptionkey@0"
I20260929 03:03:33.285715 30734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:33.298488 30734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:33.302282 30734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:33.302565 30734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:33.319983 30734 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:36349
I20260929 03:03:33.320003 30800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:36349 every 8 connection(s)
I20260929 03:03:33.321419 30734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:33.324565 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 30734
I20260929 03:03:33.324743 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:33.325053 30801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:33.325289 9025 external_mini_cluster.cc:1428] Setting key 87ca7ec4278ea4662dd489356410c1bc
I20260929 03:03:33.339741 30801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36: Bootstrap starting.
Sep 29 03:03:33 dist-test-slave-m1rv krb5kdc[30721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:03:33.343070 30801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:33.344132 30801 log.cc:824] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:33.346884 30801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36: No bootstrap required, opened a new log
I20260929 03:03:33.348708 30804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:33.327024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35409 (local address 127.8.208.126:36349)
0929 03:03:33.327818 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:33.327830 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:33.327884 (+ 54us) server_negotiation.cc:407] Connection header received
0929 03:03:33.329737 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:33.329757 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:33.330151 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:33.330486 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:33.332332 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:33.333687 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:33.335461 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:33.336155 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:33.336286 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:33.341361 (+ 5075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:33.341372 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:33.341395 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:33.341438 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:33.344603 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:33.345460 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:33.345470 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:33.345483 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:33.345640 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:33.346118 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:33.346127 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:33.346129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:33.346620 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:33.346969 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:33.347427 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 03:03:33.347738 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":379,"threads_started":1}
I20260929 03:03:33.352487 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:33.354872 30801 raft_consensus.cc:348] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b49d3396b72746beb3ee786594511a36" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36349 } }
I20260929 03:03:33.355284 30801 raft_consensus.cc:374] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:33.355417 30801 raft_consensus.cc:729] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b49d3396b72746beb3ee786594511a36, State: Initialized, Role: FOLLOWER
I20260929 03:03:33.356098 30801 consensus_queue.cc:260] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b49d3396b72746beb3ee786594511a36" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36349 } }
I20260929 03:03:33.356369 30801 raft_consensus.cc:388] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:33.356544 30801 raft_consensus.cc:482] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:33.356765 30801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:33.359063 30801 raft_consensus.cc:504] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b49d3396b72746beb3ee786594511a36" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36349 } }
I20260929 03:03:33.359591 30801 leader_election.cc:302] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b49d3396b72746beb3ee786594511a36; no voters:
I20260929 03:03:33.360245 30801 leader_election.cc:288] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:33.360347 30811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:33.361011 30811 raft_consensus.cc:686] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [term 1 LEADER]: Becoming Leader. State: Replica: b49d3396b72746beb3ee786594511a36, State: Running, Role: LEADER
I20260929 03:03:33.361789 30811 consensus_queue.cc:237] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b49d3396b72746beb3ee786594511a36" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36349 } }
I20260929 03:03:33.364017 30811 sys_catalog.cc:455] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b49d3396b72746beb3ee786594511a36. Latest consensus state: current_term: 1 leader_uuid: "b49d3396b72746beb3ee786594511a36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b49d3396b72746beb3ee786594511a36" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36349 } } }
I20260929 03:03:33.364432 30811 sys_catalog.cc:458] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:33.366616 30813 sys_catalog.cc:455] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b49d3396b72746beb3ee786594511a36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b49d3396b72746beb3ee786594511a36" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36349 } } }
I20260929 03:03:33.367126 30813 sys_catalog.cc:458] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:33.369096 30801 sys_catalog.cc:564] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:33.374238 30804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:33.356717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39873 (local address 127.8.208.126:36349)
0929 03:03:33.356880 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:33.356888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:33.357395 (+ 507us) server_negotiation.cc:407] Connection header received
0929 03:03:33.357598 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:33.357603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:33.357673 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:33.357790 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:33.359194 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:33.360076 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:33.361907 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:33.362514 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:33.362648 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:33.365737 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:33.365745 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:33.365751 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:33.365790 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:33.369040 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:33.370088 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:33.370097 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:33.370103 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:33.370350 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:33.371113 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:33.371122 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:33.371127 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:33.371608 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:33.373285 (+ 1677us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:33.373730 (+ 445us) server_negotiation.cc:299] Negotiation successful
0929 03:03:33.373912 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 03:03:33.374816 30815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:33.381305 30815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:33.390322 30815 catalog_manager.cc:1347] Generated new cluster ID: e9f090b0b2ab4a519be9194ab2b80931
I20260929 03:03:33.390529 30815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:33.414062 30815 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:33.415786 30815 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:33.425783 30815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b49d3396b72746beb3ee786594511a36: Generated new TSK 0
I20260929 03:03:33.426903 30815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:33.444736 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30734
I20260929 03:03:33.446607 30796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:33.539718 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 30734
W20260929 03:03:33.556694 30719 connection.cc:535] client connection to 127.8.208.126:36349 recv error: Network error: failed to read from TLS socket (remote: 127.8.208.126:36349): Connection reset by peer (error 104)
2026-09-29T03:03:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2026 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:33 dist-test-slave-m1rv krb5kdc[30833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:33 dist-test-slave-m1rv krb5kdc[30833](info): set up 2 sockets
Sep 29 03:03:33 dist-test-slave-m1rv krb5kdc[30833](info): commencing operation
krb5kdc: starting...
W20260929 03:03:34.657667 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:34 dist-test-slave-m1rv krb5kdc[30833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:03:34.904521 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:42657
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:44389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:42657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:35.236445 30846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:35.236821 30846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:35.237015 30846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:35.244426 30846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:35.244586 30846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:35.244705 30846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:35.244828 30846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:35.244932 30846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:35.255147 30846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:44389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:42657
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:42657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:35.256539 30846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:35.257966 30846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:35.270702 30851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:35.270744 30852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:35.270771 30854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:35.271649 30846 server_base.cc:1034] running on GCE node
I20260929 03:03:35.272418 30846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:35.273910 30846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:35.275197 30846 hybrid_clock.cc:648] HybridClock initialized: now 1790651015275154 us; error 49 us; skew 500 ppm
I20260929 03:03:35.275530 30846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:35 dist-test-slave-m1rv krb5kdc[30833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651015, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:35.278836 30846 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:03:35.280225 30846 webserver.cc:460] Webserver started at http://127.8.208.126:43107/ using document root <none> and password file <none>
I20260929 03:03:35.280848 30846 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:35.280942 30846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:35.281235 30846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:35.284114 30846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "f38a9a2b80b143cda1a44541ae3b6788"
format_stamp: "Formatted at 2026-09-29 03:03:35 on dist-test-slave-m1rv"
server_key: "79be688c3605ccfd3f0377bf0bbed598"
server_key_iv: "b1ef83ffece7926f8d843ffdf87348e3"
server_key_version: "encryptionkey@0"
I20260929 03:03:35.284832 30846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "f38a9a2b80b143cda1a44541ae3b6788"
format_stamp: "Formatted at 2026-09-29 03:03:35 on dist-test-slave-m1rv"
server_key: "79be688c3605ccfd3f0377bf0bbed598"
server_key_iv: "b1ef83ffece7926f8d843ffdf87348e3"
server_key_version: "encryptionkey@0"
I20260929 03:03:35.289685 30846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260929 03:03:35.292852 30861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:35.294184 30846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 03:03:35.294308 30846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "f38a9a2b80b143cda1a44541ae3b6788"
format_stamp: "Formatted at 2026-09-29 03:03:35 on dist-test-slave-m1rv"
server_key: "79be688c3605ccfd3f0377bf0bbed598"
server_key_iv: "b1ef83ffece7926f8d843ffdf87348e3"
server_key_version: "encryptionkey@0"
I20260929 03:03:35.294456 30846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:35.315809 30846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:35.319621 30846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:35.319844 30846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:35.338220 30846 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:42657
I20260929 03:03:35.338251 30912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:42657 every 8 connection(s)
I20260929 03:03:35.339545 30846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:35.343041 30913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:35.348104 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 30846
I20260929 03:03:35.348280 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:35.348790 9025 external_mini_cluster.cc:1428] Setting key 539442a61c2fe6d715295d952194ffb2
I20260929 03:03:35.354290 30913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788: Bootstrap starting.
I20260929 03:03:35.357772 30913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:35.358966 30913 log.cc:824] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:35.362042 30913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788: No bootstrap required, opened a new log
I20260929 03:03:35.362130 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:35.350572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59381 (local address 127.8.208.126:42657)
0929 03:03:35.351324 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:35.351335 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:35.351373 (+ 38us) server_negotiation.cc:407] Connection header received
0929 03:03:35.353073 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:35.353094 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:35.353544 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:35.353852 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:35.354455 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:35.355720 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:35.357613 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:35.358325 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:35.358443 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:35.359816 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:35.359826 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:35.359842 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:35.360545 (+ 703us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:35.360611 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:35.360633 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:35.360748 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:35.361016 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 03:03:35.361219 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":354,"threads_started":1}
I20260929 03:03:35.365816 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:35.370287 30913 raft_consensus.cc:348] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38a9a2b80b143cda1a44541ae3b6788" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42657 } }
I20260929 03:03:35.370728 30913 raft_consensus.cc:374] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:35.370836 30913 raft_consensus.cc:729] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f38a9a2b80b143cda1a44541ae3b6788, State: Initialized, Role: FOLLOWER
I20260929 03:03:35.371449 30913 consensus_queue.cc:260] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38a9a2b80b143cda1a44541ae3b6788" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42657 } }
I20260929 03:03:35.371690 30913 raft_consensus.cc:388] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:35.371835 30913 raft_consensus.cc:482] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:35.372001 30913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:35.374773 30913 raft_consensus.cc:504] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38a9a2b80b143cda1a44541ae3b6788" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42657 } }
I20260929 03:03:35.375274 30913 leader_election.cc:302] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f38a9a2b80b143cda1a44541ae3b6788; no voters:
I20260929 03:03:35.375947 30913 leader_election.cc:288] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:35.376061 30923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:35.378084 30913 sys_catalog.cc:564] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:35.381284 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:35.370135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36083 (local address 127.8.208.126:42657)
0929 03:03:35.370325 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:35.370331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:35.373753 (+ 3422us) server_negotiation.cc:407] Connection header received
0929 03:03:35.373932 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:35.373936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:35.373992 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:35.374073 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:35.374650 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:35.375467 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:35.377515 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:35.378503 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:35.378667 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:35.380180 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:35.380190 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:35.380198 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:35.380228 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:35.380314 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:35.380339 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:35.380490 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:35.380737 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 03:03:35.380807 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 03:03:35.382957 30923 raft_consensus.cc:686] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [term 1 LEADER]: Becoming Leader. State: Replica: f38a9a2b80b143cda1a44541ae3b6788, State: Running, Role: LEADER
I20260929 03:03:35.383510 30923 consensus_queue.cc:237] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38a9a2b80b143cda1a44541ae3b6788" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42657 } }
I20260929 03:03:35.386287 30925 sys_catalog.cc:455] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f38a9a2b80b143cda1a44541ae3b6788. Latest consensus state: current_term: 1 leader_uuid: "f38a9a2b80b143cda1a44541ae3b6788" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38a9a2b80b143cda1a44541ae3b6788" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42657 } } }
I20260929 03:03:35.387306 30925 sys_catalog.cc:458] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:35.387889 30924 sys_catalog.cc:455] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f38a9a2b80b143cda1a44541ae3b6788" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38a9a2b80b143cda1a44541ae3b6788" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42657 } } }
I20260929 03:03:35.388307 30924 sys_catalog.cc:458] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:35.394340 30933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:35.398737 30933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:35.407399 30933 catalog_manager.cc:1347] Generated new cluster ID: 62ca9d78ba3b4c658ae69c356453f1e7
I20260929 03:03:35.407527 30933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:35.427886 30933 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:35.429761 30933 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:35.449807 30933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f38a9a2b80b143cda1a44541ae3b6788: Generated new TSK 0
I20260929 03:03:35.450762 30933 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:35.465994 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30846
W20260929 03:03:35.467118 30908 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59381
W20260929 03:03:35.467936 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30846: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 03:03:35.468115 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 30846
W20260929 03:03:35.485105 30831 connection.cc:535] client connection to 127.8.208.126:42657 recv error: Network error: failed to read from TLS socket (remote: 127.8.208.126:42657): Connection reset by peer (error 104)
2026-09-29T03:03:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1928 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:35 dist-test-slave-m1rv krb5kdc[30944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:35 dist-test-slave-m1rv krb5kdc[30944](info): set up 2 sockets
Sep 29 03:03:35 dist-test-slave-m1rv krb5kdc[30944](info): commencing operation
krb5kdc: starting...
W20260929 03:03:36.569875 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.009s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:36 dist-test-slave-m1rv krb5kdc[30944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:03:36.821350 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:36973
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:45965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:36973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:37.147840 30957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:37.148200 30957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:37.148386 30957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:37.158385 30957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:37.158536 30957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:37.158620 30957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:37.158707 30957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:37.158793 30957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:37.168880 30957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:45965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:36973
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:36973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:37.170267 30957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:37.171567 30957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:37.183468 30963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:37.183423 30965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:37.183481 30962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:37.184166 30957 server_base.cc:1034] running on GCE node
I20260929 03:03:37.184921 30957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:37.186362 30957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:37.187582 30957 hybrid_clock.cc:648] HybridClock initialized: now 1790651017187520 us; error 52 us; skew 500 ppm
I20260929 03:03:37.187923 30957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:37 dist-test-slave-m1rv krb5kdc[30944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651017, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:37.191156 30957 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:03:37.192690 30957 webserver.cc:460] Webserver started at http://127.8.208.126:45847/ using document root <none> and password file <none>
I20260929 03:03:37.193461 30957 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:37.193612 30957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:37.193917 30957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:37.196943 30957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "757d0464fdd94d7aa528c10d619dd70b"
format_stamp: "Formatted at 2026-09-29 03:03:37 on dist-test-slave-m1rv"
server_key: "3815f21bc1fd64ceb2d5f4c269b46622"
server_key_iv: "7b87d707e27aae4cffc6f78f0004663d"
server_key_version: "encryptionkey@0"
I20260929 03:03:37.197789 30957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "757d0464fdd94d7aa528c10d619dd70b"
format_stamp: "Formatted at 2026-09-29 03:03:37 on dist-test-slave-m1rv"
server_key: "3815f21bc1fd64ceb2d5f4c269b46622"
server_key_iv: "7b87d707e27aae4cffc6f78f0004663d"
server_key_version: "encryptionkey@0"
I20260929 03:03:37.202849 30957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 03:03:37.206357 30972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:37.207703 30957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:03:37.207895 30957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "757d0464fdd94d7aa528c10d619dd70b"
format_stamp: "Formatted at 2026-09-29 03:03:37 on dist-test-slave-m1rv"
server_key: "3815f21bc1fd64ceb2d5f4c269b46622"
server_key_iv: "7b87d707e27aae4cffc6f78f0004663d"
server_key_version: "encryptionkey@0"
I20260929 03:03:37.208107 30957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:37.213784 30957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:37.217819 30957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:37.218142 30957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:37.236160 30957 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:36973
I20260929 03:03:37.236193 31023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:36973 every 8 connection(s)
I20260929 03:03:37.237617 30957 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:37.241349 31024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:37.244696 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 30957
I20260929 03:03:37.244872 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:37.245484 9025 external_mini_cluster.cc:1428] Setting key 123fd831ebd74ee498ffdee8439e4c08
I20260929 03:03:37.254508 31027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:37.247528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51117 (local address 127.8.208.126:36973)
0929 03:03:37.248506 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:37.248521 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:37.248905 (+ 384us) server_negotiation.cc:407] Connection header received
0929 03:03:37.250645 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:37.250682 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:37.251132 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:37.251587 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:37.252574 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:37.252583 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:37.252606 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:37.252632 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:37.252693 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:37.252721 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:37.252939 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:37.253067 (+ 128us) server_negotiation.cc:299] Negotiation successful
0929 03:03:37.253290 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":434,"threads_started":1}
I20260929 03:03:37.255256 31024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b: Bootstrap starting.
I20260929 03:03:37.258066 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:37.258747 31024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:37.259955 31024 log.cc:824] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:37.263255 31024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b: No bootstrap required, opened a new log
I20260929 03:03:37.265748 31027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:37.263090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48515 (local address 127.8.208.126:36973)
0929 03:03:37.263280 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:37.263285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:37.263804 (+ 519us) server_negotiation.cc:407] Connection header received
0929 03:03:37.263907 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:37.263912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:37.263972 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:37.264082 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:37.265009 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:37.265016 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:37.265033 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:37.265052 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:37.265102 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:37.265130 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:37.265311 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:37.265405 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 03:03:37.265466 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 03:03:37.271656 31024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "757d0464fdd94d7aa528c10d619dd70b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36973 } }
I20260929 03:03:37.272004 31024 raft_consensus.cc:374] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:37.272078 31024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757d0464fdd94d7aa528c10d619dd70b, State: Initialized, Role: FOLLOWER
I20260929 03:03:37.272671 31024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "757d0464fdd94d7aa528c10d619dd70b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36973 } }
I20260929 03:03:37.272923 31024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:37.273084 31024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:37.273294 31024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:37.275451 31024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "757d0464fdd94d7aa528c10d619dd70b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36973 } }
I20260929 03:03:37.275943 31024 leader_election.cc:302] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 757d0464fdd94d7aa528c10d619dd70b; no voters:
I20260929 03:03:37.276503 31024 leader_election.cc:288] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:37.276654 31034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:37.277359 31034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [term 1 LEADER]: Becoming Leader. State: Replica: 757d0464fdd94d7aa528c10d619dd70b, State: Running, Role: LEADER
I20260929 03:03:37.277845 31034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "757d0464fdd94d7aa528c10d619dd70b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36973 } }
I20260929 03:03:37.278568 31024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:37.280835 31036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 757d0464fdd94d7aa528c10d619dd70b. Latest consensus state: current_term: 1 leader_uuid: "757d0464fdd94d7aa528c10d619dd70b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "757d0464fdd94d7aa528c10d619dd70b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36973 } } }
I20260929 03:03:37.281421 31036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:37.280869 31035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "757d0464fdd94d7aa528c10d619dd70b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "757d0464fdd94d7aa528c10d619dd70b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36973 } } }
I20260929 03:03:37.282096 31035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:37.283370 31041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:37.288632 31041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:37.297372 31041 catalog_manager.cc:1347] Generated new cluster ID: 36c9236f320d45b3a8b6bb502a61a174
I20260929 03:03:37.297514 31041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:37.313360 31041 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:37.313868 31041 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:37.328038 31041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 757d0464fdd94d7aa528c10d619dd70b: Generated new TSK 0
I20260929 03:03:37.329056 31041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:37.338779 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30957
W20260929 03:03:37.340361 31019 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51117
W20260929 03:03:37.341099 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 30957: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 03:03:37.341264 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 30957
2026-09-29T03:03:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1872 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:37 dist-test-slave-m1rv krb5kdc[31055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:37 dist-test-slave-m1rv krb5kdc[31055](info): set up 2 sockets
Sep 29 03:03:37 dist-test-slave-m1rv krb5kdc[31055](info): commencing operation
krb5kdc: starting...
W20260929 03:03:38.442016 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:38 dist-test-slave-m1rv krb5kdc[31055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:03:38.704329 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:36387
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:37739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:36387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:39.039714 31068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:39.040002 31068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:39.040105 31068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:39.047269 31068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:39.047387 31068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:39.047434 31068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:39.047494 31068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:39.047544 31068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:39.057219 31068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:37739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:36387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:36387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:39.058410 31068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:39.059605 31068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:39.072636 31076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:39.072687 31073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:39.072575 31074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:39.078639 31068 server_base.cc:1034] running on GCE node
I20260929 03:03:39.079447 31068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:39.080911 31068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:39.082146 31068 hybrid_clock.cc:648] HybridClock initialized: now 1790651019082076 us; error 63 us; skew 500 ppm
I20260929 03:03:39.082501 31068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:39 dist-test-slave-m1rv krb5kdc[31055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651019, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:39.095772 31068 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:03:39.097975 31068 webserver.cc:460] Webserver started at http://127.8.208.126:46867/ using document root <none> and password file <none>
I20260929 03:03:39.098686 31068 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:39.098788 31068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:39.099041 31068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:39.102034 31068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "4a27282f899c45f5b35bd1d13a285c00"
format_stamp: "Formatted at 2026-09-29 03:03:39 on dist-test-slave-m1rv"
server_key: "476b2624e5f5a13857204eca8497f296"
server_key_iv: "f530cf72b2ff1bd05ce284e8e9178f13"
server_key_version: "encryptionkey@0"
I20260929 03:03:39.102746 31068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "4a27282f899c45f5b35bd1d13a285c00"
format_stamp: "Formatted at 2026-09-29 03:03:39 on dist-test-slave-m1rv"
server_key: "476b2624e5f5a13857204eca8497f296"
server_key_iv: "f530cf72b2ff1bd05ce284e8e9178f13"
server_key_version: "encryptionkey@0"
I20260929 03:03:39.107777 31068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260929 03:03:39.111269 31083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:39.112766 31068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:03:39.112957 31068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "4a27282f899c45f5b35bd1d13a285c00"
format_stamp: "Formatted at 2026-09-29 03:03:39 on dist-test-slave-m1rv"
server_key: "476b2624e5f5a13857204eca8497f296"
server_key_iv: "f530cf72b2ff1bd05ce284e8e9178f13"
server_key_version: "encryptionkey@0"
I20260929 03:03:39.113168 31068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:39.136193 31068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:39.140892 31068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:39.141211 31068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:39.160521 31068 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:36387
I20260929 03:03:39.160552 31134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:36387 every 8 connection(s)
I20260929 03:03:39.161948 31068 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:39.165710 31135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:39.169936 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 31068
I20260929 03:03:39.170132 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:39.170581 9025 external_mini_cluster.cc:1428] Setting key 6d410c0ecfdf8b127d0a64e0aebdd8bc
I20260929 03:03:39.176925 31135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00: Bootstrap starting.
I20260929 03:03:39.180581 31135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:39.181842 31135 log.cc:824] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:39.185457 31135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00: No bootstrap required, opened a new log
Sep 29 03:03:39 dist-test-slave-m1rv krb5kdc[31055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:03:39.193403 31135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a27282f899c45f5b35bd1d13a285c00" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36387 } }
I20260929 03:03:39.193920 31135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:39.194060 31135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a27282f899c45f5b35bd1d13a285c00, State: Initialized, Role: FOLLOWER
I20260929 03:03:39.194713 31135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a27282f899c45f5b35bd1d13a285c00" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36387 } }
I20260929 03:03:39.194947 31135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:39.195118 31135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:39.195158 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:39.172477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42237 (local address 127.8.208.126:36387)
0929 03:03:39.173357 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:39.173368 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:39.173409 (+ 41us) server_negotiation.cc:407] Connection header received
0929 03:03:39.175296 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:39.175320 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:39.175724 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:39.176057 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:39.177881 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:39.179182 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:39.180963 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:39.181735 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:39.181869 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:39.186744 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:39.186752 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:39.186769 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:39.186804 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:39.190573 (+ 3769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:39.191318 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:39.191325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:39.191332 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:39.191499 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:39.191915 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:39.191926 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:39.191929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:39.192500 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:39.192815 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:39.193326 (+ 511us) server_negotiation.cc:299] Negotiation successful
0929 03:03:39.193784 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":432,"threads_started":1}
I20260929 03:03:39.195310 31135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:39.197836 31135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a27282f899c45f5b35bd1d13a285c00" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36387 } }
I20260929 03:03:39.198383 31135 leader_election.cc:302] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a27282f899c45f5b35bd1d13a285c00; no voters:
I20260929 03:03:39.198975 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:39.199154 31135 leader_election.cc:288] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:39.199244 31140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:39.201094 31135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:39.206475 31140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [term 1 LEADER]: Becoming Leader. State: Replica: 4a27282f899c45f5b35bd1d13a285c00, State: Running, Role: LEADER
I20260929 03:03:39.207427 31140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a27282f899c45f5b35bd1d13a285c00" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36387 } }
I20260929 03:03:39.211362 31142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a27282f899c45f5b35bd1d13a285c00. Latest consensus state: current_term: 1 leader_uuid: "4a27282f899c45f5b35bd1d13a285c00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a27282f899c45f5b35bd1d13a285c00" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36387 } } }
I20260929 03:03:39.211776 31142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:39.211416 31141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a27282f899c45f5b35bd1d13a285c00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a27282f899c45f5b35bd1d13a285c00" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 36387 } } }
I20260929 03:03:39.212273 31141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:39.223484 31159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:39.229385 31159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:39.229947 31138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:39.209652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35553 (local address 127.8.208.126:36387)
0929 03:03:39.209835 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:39.209843 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:39.209975 (+ 132us) server_negotiation.cc:407] Connection header received
0929 03:03:39.210194 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:39.210200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:39.210289 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:39.210402 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:39.212710 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:39.214275 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:39.216818 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:39.217942 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:39.218088 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:39.222623 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:39.222633 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:39.222642 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:39.222701 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:39.226652 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:39.227476 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:39.227481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:39.227487 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:39.227637 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:39.228113 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:39.228124 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:39.228129 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:39.228629 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:39.228889 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:39.229334 (+ 445us) server_negotiation.cc:299] Negotiation successful
0929 03:03:39.229581 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 03:03:39.238994 31159 catalog_manager.cc:1347] Generated new cluster ID: 017e715e1097434e90ae737cf3e2f295
I20260929 03:03:39.239127 31159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:39.252126 31159 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:39.253508 31159 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:39.267052 31159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a27282f899c45f5b35bd1d13a285c00: Generated new TSK 0
I20260929 03:03:39.267900 31159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:39.273324 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 31068
I20260929 03:03:39.274473 31130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:39.364121 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 31068
2026-09-29T03:03:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2022 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:39 dist-test-slave-m1rv krb5kdc[31167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:39 dist-test-slave-m1rv krb5kdc[31167](info): set up 2 sockets
Sep 29 03:03:39 dist-test-slave-m1rv krb5kdc[31167](info): commencing operation
krb5kdc: starting...
W20260929 03:03:40.469640 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:40 dist-test-slave-m1rv krb5kdc[31167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:03:40.716711 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:35745
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:33679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:35745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:41.041633 31180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:41.041904 31180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:41.042004 31180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:41.049113 31180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:41.049232 31180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:41.049279 31180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:41.049338 31180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:41.049399 31180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:41.058837 31180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:33679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:35745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:35745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:41.059958 31180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:41.061147 31180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:41.072753 31186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:41.072850 31188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:41.072803 31185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:41.073890 31180 server_base.cc:1034] running on GCE node
I20260929 03:03:41.074661 31180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:41.076081 31180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:41.077301 31180 hybrid_clock.cc:648] HybridClock initialized: now 1790651021077260 us; error 35 us; skew 500 ppm
I20260929 03:03:41.077667 31180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:41 dist-test-slave-m1rv krb5kdc[31167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651021, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:41.080729 31180 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:03:41.082211 31180 webserver.cc:460] Webserver started at http://127.8.208.126:45045/ using document root <none> and password file <none>
I20260929 03:03:41.082864 31180 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:41.083057 31180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:41.083374 31180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:41.086313 31180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "29a3efe3b4ea46669d821b11ce8a7c6c"
format_stamp: "Formatted at 2026-09-29 03:03:41 on dist-test-slave-m1rv"
server_key: "a8f982f6cc8d615518460f0163489a03"
server_key_iv: "f3d4b0232aef310e5133584675e16189"
server_key_version: "encryptionkey@0"
I20260929 03:03:41.087064 31180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "29a3efe3b4ea46669d821b11ce8a7c6c"
format_stamp: "Formatted at 2026-09-29 03:03:41 on dist-test-slave-m1rv"
server_key: "a8f982f6cc8d615518460f0163489a03"
server_key_iv: "f3d4b0232aef310e5133584675e16189"
server_key_version: "encryptionkey@0"
I20260929 03:03:41.092008 31180 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 03:03:41.095288 31195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:41.096868 31180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 03:03:41.097062 31180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "29a3efe3b4ea46669d821b11ce8a7c6c"
format_stamp: "Formatted at 2026-09-29 03:03:41 on dist-test-slave-m1rv"
server_key: "a8f982f6cc8d615518460f0163489a03"
server_key_iv: "f3d4b0232aef310e5133584675e16189"
server_key_version: "encryptionkey@0"
I20260929 03:03:41.097332 31180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:41.108070 31180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:41.111874 31180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:41.112164 31180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:41.130357 31180 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:35745
I20260929 03:03:41.130376 31246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:35745 every 8 connection(s)
I20260929 03:03:41.131767 31180 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:41.135341 31247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:41.139981 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 31180
I20260929 03:03:41.140167 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:41.140717 9025 external_mini_cluster.cc:1428] Setting key 82d3a8dce6a74b7f326c252b4962b029
I20260929 03:03:41.146776 31247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c: Bootstrap starting.
I20260929 03:03:41.150292 31247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:41.151507 31247 log.cc:824] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:41.154459 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:41.142637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55725 (local address 127.8.208.126:35745)
0929 03:03:41.143458 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:41.143469 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:41.143510 (+ 41us) server_negotiation.cc:407] Connection header received
0929 03:03:41.145398 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:41.145427 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:41.145805 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:41.146130 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:41.146726 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:41.148075 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:41.149932 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:41.150656 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:41.150789 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:41.151954 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:41.151961 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:41.151974 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:41.152773 (+ 799us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:41.152863 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:41.152898 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:41.152997 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:41.153287 (+ 290us) server_negotiation.cc:299] Negotiation successful
0929 03:03:41.153577 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":405,"threads_started":1}
I20260929 03:03:41.154593 31247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c: No bootstrap required, opened a new log
I20260929 03:03:41.157956 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:41.164356 31247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a3efe3b4ea46669d821b11ce8a7c6c" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35745 } }
I20260929 03:03:41.164790 31247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:41.164896 31247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a3efe3b4ea46669d821b11ce8a7c6c, State: Initialized, Role: FOLLOWER
I20260929 03:03:41.165609 31247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a3efe3b4ea46669d821b11ce8a7c6c" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35745 } }
I20260929 03:03:41.165931 31247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:41.166090 31247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:41.166260 31247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:41.169243 31247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a3efe3b4ea46669d821b11ce8a7c6c" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35745 } }
I20260929 03:03:41.169780 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:41.163164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40813 (local address 127.8.208.126:35745)
0929 03:03:41.163352 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:41.163358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:41.163807 (+ 449us) server_negotiation.cc:407] Connection header received
0929 03:03:41.163962 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:41.163967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:41.164047 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:41.164152 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:41.164639 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:41.165712 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:41.167201 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:41.167895 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:41.168030 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:41.169014 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:41.169022 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:41.169029 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:41.169053 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:41.169122 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:41.169143 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:41.169293 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:41.169382 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 03:03:41.169469 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 03:03:41.169822 31247 leader_election.cc:302] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29a3efe3b4ea46669d821b11ce8a7c6c; no voters:
I20260929 03:03:41.170456 31247 leader_election.cc:288] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:41.170612 31257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:41.171869 31257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [term 1 LEADER]: Becoming Leader. State: Replica: 29a3efe3b4ea46669d821b11ce8a7c6c, State: Running, Role: LEADER
I20260929 03:03:41.172380 31257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a3efe3b4ea46669d821b11ce8a7c6c" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35745 } }
I20260929 03:03:41.172659 31247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:41.175562 31259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29a3efe3b4ea46669d821b11ce8a7c6c. Latest consensus state: current_term: 1 leader_uuid: "29a3efe3b4ea46669d821b11ce8a7c6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a3efe3b4ea46669d821b11ce8a7c6c" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35745 } } }
I20260929 03:03:41.175931 31259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:41.176512 31258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29a3efe3b4ea46669d821b11ce8a7c6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29a3efe3b4ea46669d821b11ce8a7c6c" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 35745 } } }
I20260929 03:03:41.176941 31258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:41.177006 31262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:41.182981 31262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:41.193310 31262 catalog_manager.cc:1347] Generated new cluster ID: 2c04b819dbdf4f1eb6908364a5050937
I20260929 03:03:41.193456 31262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:41.212882 31262 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:41.214252 31262 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:41.225916 31262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29a3efe3b4ea46669d821b11ce8a7c6c: Generated new TSK 0
I20260929 03:03:41.226722 31262 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:41.238343 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 31180
W20260929 03:03:41.239507 31242 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55725
W20260929 03:03:41.240175 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 31180: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 03:03:41.240329 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 31180
2026-09-29T03:03:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1872 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:41 dist-test-slave-m1rv krb5kdc[31279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:41 dist-test-slave-m1rv krb5kdc[31279](info): set up 2 sockets
Sep 29 03:03:41 dist-test-slave-m1rv krb5kdc[31279](info): commencing operation
krb5kdc: starting...
W20260929 03:03:42.341773 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:42 dist-test-slave-m1rv krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:03:42.596794 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:42079
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:37817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:42079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:42.929807 31292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:42.930097 31292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:42.930285 31292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:42.937645 31292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:42.937780 31292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:42.937891 31292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:42.937990 31292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:42.938073 31292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:42.947935 31292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:37817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:42079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:42079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:42.949297 31292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:42.950610 31292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:42.962360 31300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:42.962473 31298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:42.962445 31297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:42.963488 31292 server_base.cc:1034] running on GCE node
I20260929 03:03:42.964174 31292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:42.965523 31292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:42.966704 31292 hybrid_clock.cc:648] HybridClock initialized: now 1790651022966659 us; error 38 us; skew 500 ppm
I20260929 03:03:42.967002 31292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:42 dist-test-slave-m1rv krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651022, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:42.970145 31292 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:03:42.971648 31292 webserver.cc:460] Webserver started at http://127.8.208.126:40397/ using document root <none> and password file <none>
I20260929 03:03:42.972303 31292 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:42.972447 31292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:42.972790 31292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:42.975733 31292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "2dfb84d9dae14743a39fc3830fb050ba"
format_stamp: "Formatted at 2026-09-29 03:03:42 on dist-test-slave-m1rv"
server_key: "4dc01a8bbf2798469a3a22a5863ed21c"
server_key_iv: "9a940025ab0aaf4d822a32d6b85bcf3b"
server_key_version: "encryptionkey@0"
I20260929 03:03:42.976496 31292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "2dfb84d9dae14743a39fc3830fb050ba"
format_stamp: "Formatted at 2026-09-29 03:03:42 on dist-test-slave-m1rv"
server_key: "4dc01a8bbf2798469a3a22a5863ed21c"
server_key_iv: "9a940025ab0aaf4d822a32d6b85bcf3b"
server_key_version: "encryptionkey@0"
I20260929 03:03:42.981660 31292 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 03:03:42.984926 31307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:42.986435 31292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:03:42.986624 31292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "2dfb84d9dae14743a39fc3830fb050ba"
format_stamp: "Formatted at 2026-09-29 03:03:42 on dist-test-slave-m1rv"
server_key: "4dc01a8bbf2798469a3a22a5863ed21c"
server_key_iv: "9a940025ab0aaf4d822a32d6b85bcf3b"
server_key_version: "encryptionkey@0"
I20260929 03:03:42.986816 31292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:42.992401 31292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:42.996662 31292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:42.996949 31292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:43.015283 31292 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:42079
I20260929 03:03:43.015308 31358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:42079 every 8 connection(s)
I20260929 03:03:43.016661 31292 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:43.020352 31359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:43.021518 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 31292
I20260929 03:03:43.021675 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:43.022147 9025 external_mini_cluster.cc:1428] Setting key 67ea30a1950db26cb010088fac14f836
I20260929 03:03:43.029700 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:43.023945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53485 (local address 127.8.208.126:42079)
0929 03:03:43.024685 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:43.024696 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:43.024736 (+ 40us) server_negotiation.cc:407] Connection header received
0929 03:03:43.026398 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:43.026423 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:43.026747 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:43.027146 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:43.028147 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:43.028153 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:43.028166 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:43.028180 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:43.028236 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:43.028258 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:43.028480 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:43.028611 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 03:03:43.028786 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":333,"threads_started":1}
I20260929 03:03:43.033209 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:43.033210 31359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba: Bootstrap starting.
I20260929 03:03:43.036378 31359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:43.037571 31359 log.cc:824] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:43.040127 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:43.037479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50029 (local address 127.8.208.126:42079)
0929 03:03:43.037690 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:43.037699 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:43.038254 (+ 555us) server_negotiation.cc:407] Connection header received
0929 03:03:43.038440 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:43.038445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:43.038507 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:43.038619 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:43.039523 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:43.039529 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:43.039537 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:03:43.039556 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:43.039595 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:03:43.039609 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:43.039724 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:43.039817 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 03:03:43.039871 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:03:43.040511 31359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba: No bootstrap required, opened a new log
I20260929 03:03:43.048004 31359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dfb84d9dae14743a39fc3830fb050ba" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42079 } }
I20260929 03:03:43.048373 31359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:03:43.048484 31359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dfb84d9dae14743a39fc3830fb050ba, State: Initialized, Role: FOLLOWER
I20260929 03:03:43.049036 31359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dfb84d9dae14743a39fc3830fb050ba" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42079 } }
I20260929 03:03:43.049366 31359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:03:43.049512 31359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:03:43.049691 31359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:43.051815 31359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dfb84d9dae14743a39fc3830fb050ba" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42079 } }
I20260929 03:03:43.052317 31359 leader_election.cc:302] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dfb84d9dae14743a39fc3830fb050ba; no voters:
I20260929 03:03:43.052922 31359 leader_election.cc:288] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:03:43.053048 31369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:43.054363 31369 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [term 1 LEADER]: Becoming Leader. State: Replica: 2dfb84d9dae14743a39fc3830fb050ba, State: Running, Role: LEADER
I20260929 03:03:43.054819 31369 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dfb84d9dae14743a39fc3830fb050ba" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42079 } }
I20260929 03:03:43.055055 31359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:43.059494 31370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dfb84d9dae14743a39fc3830fb050ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dfb84d9dae14743a39fc3830fb050ba" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42079 } } }
I20260929 03:03:43.059409 31371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dfb84d9dae14743a39fc3830fb050ba. Latest consensus state: current_term: 1 leader_uuid: "2dfb84d9dae14743a39fc3830fb050ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dfb84d9dae14743a39fc3830fb050ba" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 42079 } } }
I20260929 03:03:43.059902 31371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:43.059947 31370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:43.071806 31375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:43.078099 31375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:43.091107 31375 catalog_manager.cc:1347] Generated new cluster ID: 3c1d25ad399641c484e56db97148868c
I20260929 03:03:43.091241 31375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:43.115022 31375 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:43.115526 31375 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:43.142273 31375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dfb84d9dae14743a39fc3830fb050ba: Generated new TSK 0
I20260929 03:03:43.143160 31375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:43.154505 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 31292
W20260929 03:03:43.155540 31354 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53485
W20260929 03:03:43.156352 9025 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 31292: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 03:03:43.156492 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 31292
2026-09-29T03:03:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1916 ms)
[----------] 18 tests from AuthTokenIssuingTest (35071 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:43 dist-test-slave-m1rv krb5kdc[31390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:43 dist-test-slave-m1rv krb5kdc[31390](info): set up 2 sockets
Sep 29 03:03:43 dist-test-slave-m1rv krb5kdc[31390](info): commencing operation
krb5kdc: starting...
W20260929 03:03:44.257899 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.008s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:44 dist-test-slave-m1rv krb5kdc[31390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:03:44.525749 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:34389
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:34389,127.8.208.125:41127,127.8.208.124:34227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:44.865994 31403 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 03:03:44.866413 31403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:44.866549 31403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:44.866721 31403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:44.874462 31403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:44.874598 31403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:44.874722 31403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:44.874823 31403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:44.874955 31403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:44.885267 31403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:34389,127.8.208.125:41127,127.8.208.124:34227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:34389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:44.886669 31403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:44.888056 31403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:44.901032 31411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:44.901093 31409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:44.901151 31408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:44.902323 31403 server_base.cc:1034] running on GCE node
I20260929 03:03:44.903115 31403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:44.904466 31403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:44.905716 31403 hybrid_clock.cc:648] HybridClock initialized: now 1790651024905653 us; error 40 us; skew 500 ppm
I20260929 03:03:44.906051 31403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:44 dist-test-slave-m1rv krb5kdc[31390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651024, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:44.914637 31403 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:03:44.916132 31403 webserver.cc:460] Webserver started at http://127.8.208.126:42183/ using document root <none> and password file <none>
I20260929 03:03:44.916854 31403 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:44.916992 31403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:44.917369 31403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:44.920365 31403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "2c4dc94a3f9944bcb2556b937b581ad4"
format_stamp: "Formatted at 2026-09-29 03:03:44 on dist-test-slave-m1rv"
server_key: "8b7fba8e49ebd0422e85c6cd46b563f4"
server_key_iv: "71d0832f8ed028dc0c7cc5515204a29d"
server_key_version: "encryptionkey@0"
I20260929 03:03:44.921126 31403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "2c4dc94a3f9944bcb2556b937b581ad4"
format_stamp: "Formatted at 2026-09-29 03:03:44 on dist-test-slave-m1rv"
server_key: "8b7fba8e49ebd0422e85c6cd46b563f4"
server_key_iv: "71d0832f8ed028dc0c7cc5515204a29d"
server_key_version: "encryptionkey@0"
I20260929 03:03:44.926327 31403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 03:03:44.929752 31418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:44.931020 31403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:03:44.931198 31403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "2c4dc94a3f9944bcb2556b937b581ad4"
format_stamp: "Formatted at 2026-09-29 03:03:44 on dist-test-slave-m1rv"
server_key: "8b7fba8e49ebd0422e85c6cd46b563f4"
server_key_iv: "71d0832f8ed028dc0c7cc5515204a29d"
server_key_version: "encryptionkey@0"
I20260929 03:03:44.931411 31403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:44.968973 31403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:44.973656 31403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:44.973887 31403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:44.992777 31403 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:34389
I20260929 03:03:44.992797 31469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:34389 every 8 connection(s)
I20260929 03:03:44.994182 31403 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:44.997964 31470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:45.001360 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 31403
I20260929 03:03:45.001595 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:45.002274 9025 external_mini_cluster.cc:1428] Setting key a15590a463c1fa6804afece76c9f49de
I20260929 03:03:45.004630 31470 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:03:45 dist-test-slave-m1rv krb5kdc[31390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651024, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.208.125@KRBTEST.COM; defaulting to no policy
I20260929 03:03:45.028519 31473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.009485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59662 (local address 127.8.208.126:34389)
0929 03:03:45.010220 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:45.010231 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:45.010295 (+ 64us) server_negotiation.cc:407] Connection header received
0929 03:03:45.012102 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:45.012107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:45.012343 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:45.012570 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:45.013920 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.015088 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.016467 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.017035 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.017300 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:45.020870 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:45.020878 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:45.020889 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:45.020922 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:45.023952 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:45.024644 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:45.024650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:45.024652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:45.024804 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:45.025245 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:45.025249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:45.025250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:45.025729 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:45.025949 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:45.026449 (+ 500us) server_negotiation.cc:299] Negotiation successful
0929 03:03:45.026831 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":552,"thread_start_us":358,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.125@KRBTEST.COM" created.
I20260929 03:03:45.028558 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.008135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:34389 (local address 127.0.0.1:59662)
0929 03:03:45.008919 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.008960 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:45.011676 (+ 2716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:45.012575 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:45.012581 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:45.013370 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:45.013707 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:45.013723 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.015247 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.015252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:45.016326 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:45.016338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.017316 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.017323 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:45.017513 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:45.018191 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:45.018214 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:45.020634 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:45.024084 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:45.024097 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:45.024108 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:45.024502 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:45.024928 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:45.024935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:45.024938 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:45.025102 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:45.025873 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:45.025881 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:45.026103 (+ 222us) client_negotiation.cc:769] Sending connection context
0929 03:03:45.026355 (+ 252us) client_negotiation.cc:240] Negotiation successful
0929 03:03:45.026947 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":148,"thread_start_us":338,"threads_started":1}
I20260929 03:03:45.031821 31470 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:03:45.033003 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.032486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44820)
0929 03:03:45.032689 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.032828 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260929 03:03:45.033303 31420 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
I20260929 03:03:45.034992 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.034536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44826)
0929 03:03:45.034721 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.034809 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260929 03:03:45.035197 31470 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:41127: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.125.keytab.
Entry for principal kudu/127.8.208.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.125.keytab.
I20260929 03:03:45.059835 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.125:41127
--webserver_interface=127.8.208.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.125.keytab
--principal=kudu/127.8.208.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:34389,127.8.208.125:41127,127.8.208.124:34227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:03:45.084571 31470 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } attempt: 1
I20260929 03:03:45.085789 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.085331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44830)
0929 03:03:45.085531 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.085642 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 03:03:45.086864 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.086466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44844)
0929 03:03:45.086616 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.086701 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 03:03:45.087117 31470 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:41127: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
I20260929 03:03:45.155409 31470 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } attempt: 2
I20260929 03:03:45.156713 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.156114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44850)
0929 03:03:45.156383 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.156510 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260929 03:03:45.157814 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.157413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44854)
0929 03:03:45.157564 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.157661 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 03:03:45.158007 31470 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:41127: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
I20260929 03:03:45.249310 31470 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } attempt: 3
I20260929 03:03:45.250695 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.250096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44866)
0929 03:03:45.250370 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.250508 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260929 03:03:45.251820 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.251381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44874)
0929 03:03:45.251546 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.251643 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260929 03:03:45.252022 31470 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:41127: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
W20260929 03:03:45.386389 31477 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 03:03:45.386740 31477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:45.386888 31477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:45.386987 31477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:45.394484 31477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:45.394589 31477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:45.394693 31477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:45.394783 31477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:45.394906 31477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:45.395285 31470 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } attempt: 4
I20260929 03:03:45.396522 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.395974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44886)
0929 03:03:45.396297 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.396385 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260929 03:03:45.397670 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.397318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44898)
0929 03:03:45.397466 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.397552 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 03:03:45.397850 31470 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:41127: Network error: Client connection negotiation failed: client connection to 127.8.208.125:41127: connect: Connection refused (error 111)
I20260929 03:03:45.405938 31477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:34389,127.8.208.125:41127,127.8.208.124:34227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.125.keytab
--principal=kudu/127.8.208.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.125:41127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.208.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:45.407289 31477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:45.408649 31477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:45.420905 31485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:45.420984 31482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:45.420984 31483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:45.426988 31477 server_base.cc:1034] running on GCE node
I20260929 03:03:45.427878 31477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:45.429484 31477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:45.430785 31477 hybrid_clock.cc:648] HybridClock initialized: now 1790651025430703 us; error 52 us; skew 500 ppm
I20260929 03:03:45.431195 31477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:45 dist-test-slave-m1rv krb5kdc[31390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651025, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:45.434834 31477 init.cc:377] Logged in from keytab as kudu/127.8.208.125@KRBTEST.COM (short username kudu)
I20260929 03:03:45.436596 31477 webserver.cc:460] Webserver started at http://127.8.208.125:44581/ using document root <none> and password file <none>
I20260929 03:03:45.437376 31477 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:45.437467 31477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:45.437728 31477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:45.441032 31477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/data/instance:
uuid: "47e1f269c5244196b09e6e4c4663a02b"
format_stamp: "Formatted at 2026-09-29 03:03:45 on dist-test-slave-m1rv"
server_key: "112f81d5af13d971d2e3c40878141767"
server_key_iv: "ef11b0986c31743c23bbdbf006d3cc9b"
server_key_version: "encryptionkey@0"
I20260929 03:03:45.441838 31477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/wal/instance:
uuid: "47e1f269c5244196b09e6e4c4663a02b"
format_stamp: "Formatted at 2026-09-29 03:03:45 on dist-test-slave-m1rv"
server_key: "112f81d5af13d971d2e3c40878141767"
server_key_iv: "ef11b0986c31743c23bbdbf006d3cc9b"
server_key_version: "encryptionkey@0"
I20260929 03:03:45.446981 31477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 03:03:45.450344 31492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:45.451663 31477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:03:45.451804 31477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/wal
uuid: "47e1f269c5244196b09e6e4c4663a02b"
format_stamp: "Formatted at 2026-09-29 03:03:45 on dist-test-slave-m1rv"
server_key: "112f81d5af13d971d2e3c40878141767"
server_key_iv: "ef11b0986c31743c23bbdbf006d3cc9b"
server_key_version: "encryptionkey@0"
I20260929 03:03:45.451977 31477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:45.466657 31477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:45.470629 31477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:45.470836 31477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:45.489527 31477 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.125:41127
I20260929 03:03:45.489562 31543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.125:41127 every 8 connection(s)
I20260929 03:03:45.490867 31477 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
I20260929 03:03:45.493402 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 31477
I20260929 03:03:45.493604 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-1/wal/instance
I20260929 03:03:45.494051 9025 external_mini_cluster.cc:1428] Setting key 3b05abff8539f35bf8c9ee22523e3d4d
I20260929 03:03:45.494755 31544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:45.501307 31544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:03:45 dist-test-slave-m1rv krb5kdc[31390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651025, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.125@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.208.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.124@KRBTEST.COM" created.
I20260929 03:03:45.523694 31473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.504208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59670 (local address 127.8.208.126:34389)
0929 03:03:45.504432 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:45.504438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:45.505782 (+ 1344us) server_negotiation.cc:407] Connection header received
0929 03:03:45.507696 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:45.507705 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:45.507802 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:45.507925 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:45.509513 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.510680 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.512286 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.513111 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.513292 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:45.517294 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:45.517299 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:45.517302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:45.517337 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:45.520313 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:45.521061 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:45.521070 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:45.521076 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:45.521255 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:45.521682 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:45.521689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:45.521695 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:45.522105 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:45.522264 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:45.522973 (+ 709us) server_negotiation.cc:299] Negotiation successful
0929 03:03:45.523236 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 03:03:45.524405 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.504746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:34389 (local address 127.0.0.1:59670)
0929 03:03:45.505585 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.505638 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:45.507409 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:45.508138 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:45.508146 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:45.508665 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:45.509331 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:45.509352 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.510854 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.510858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:45.512124 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:45.512142 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.513388 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.513393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:45.513603 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:45.514358 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:45.514381 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:45.516991 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:45.520475 (+ 3484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:45.520482 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:45.520497 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:45.520926 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:45.521363 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:45.521366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:45.521368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:45.521567 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:45.522242 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:45.522248 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:45.522598 (+ 350us) client_negotiation.cc:769] Sending connection context
0929 03:03:45.522846 (+ 248us) client_negotiation.cc:240] Negotiation successful
0929 03:03:45.523252 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":386,"threads_started":1}
I20260929 03:03:45.526166 31544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:03:45 dist-test-slave-m1rv krb5kdc[31390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651025, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.125@KRBTEST.COM for kudu/127.8.208.125@KRBTEST.COM
I20260929 03:03:45.543043 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.527134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.8.208.125:41127)
0929 03:03:45.527776 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:45.527786 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:45.527821 (+ 35us) server_negotiation.cc:407] Connection header received
0929 03:03:45.527882 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:45.527893 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:45.528138 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:45.528303 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:45.529097 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.530204 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.532118 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.532674 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.532775 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:45.536778 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:45.536790 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:45.536804 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:45.536832 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:45.539843 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:45.540705 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:45.540712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:45.540715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:45.540824 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:45.541324 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:45.541328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:45.541330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:45.541723 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:45.541942 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:45.542395 (+ 453us) server_negotiation.cc:299] Negotiation successful
0929 03:03:45.542734 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":129,"server-negotiator.queue_time_us":441,"thread_start_us":291,"threads_started":1}
I20260929 03:03:45.543135 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.526858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44900)
0929 03:03:45.527059 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.527078 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:45.527178 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:45.528297 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:45.528300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:45.528689 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:45.528921 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:45.528933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.530348 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.530357 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:45.531959 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:45.531976 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.532831 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.532839 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:45.533071 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:45.533715 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:45.533742 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:45.536552 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:45.539978 (+ 3426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:45.539987 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:45.539994 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:45.540576 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:45.540954 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:45.540962 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:45.540969 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:45.541133 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:45.541884 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:45.541893 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:45.542096 (+ 203us) client_negotiation.cc:769] Sending connection context
0929 03:03:45.542264 (+ 168us) client_negotiation.cc:240] Negotiation successful
0929 03:03:45.542615 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 03:03:45.545640 31544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:03:45.546963 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.546453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:53982)
0929 03:03:45.546646 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.546749 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260929 03:03:45.547240 31496 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
I20260929 03:03:45.548887 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.548577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:53984)
0929 03:03:45.548708 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.548785 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 03:03:45.549146 31544 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:34227: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.124.keytab.
Entry for principal kudu/127.8.208.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.124.keytab.
I20260929 03:03:45.556828 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.124:34227
--webserver_interface=127.8.208.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.124.keytab
--principal=kudu/127.8.208.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:41205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:34389,127.8.208.125:41127,127.8.208.124:34227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:03:45.598527 31544 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } attempt: 1
I20260929 03:03:45.599786 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.599239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54000)
0929 03:03:45.599533 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.599633 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
I20260929 03:03:45.600759 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.600387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54012)
0929 03:03:45.600546 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.600655 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 03:03:45.600978 31544 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:34227: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
I20260929 03:03:45.669210 31544 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } attempt: 2
I20260929 03:03:45.670343 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.669868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54024)
0929 03:03:45.670092 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.670190 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260929 03:03:45.671312 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.670990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54036)
0929 03:03:45.671133 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.671205 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 03:03:45.671519 31544 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:34227: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
I20260929 03:03:45.697139 31470 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } attempt: 5
Sep 29 03:03:45 dist-test-slave-m1rv krb5kdc[31390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651024, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for kudu/127.8.208.125@KRBTEST.COM
I20260929 03:03:45.710809 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.697926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44902)
0929 03:03:45.698124 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.698152 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:45.698297 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:45.698651 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:45.698656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:45.699062 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:45.699240 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:45.699250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.700241 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.700247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:45.701112 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:45.701123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.701973 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.701979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:45.702182 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:45.702875 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:45.702893 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:45.705252 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:45.708330 (+ 3078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:45.708336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:45.708340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:45.708706 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:45.709092 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:45.709096 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:45.709100 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:45.709303 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:45.709818 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:45.709823 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:45.709980 (+ 157us) client_negotiation.cc:769] Sending connection context
0929 03:03:45.710135 (+ 155us) client_negotiation.cc:240] Negotiation successful
0929 03:03:45.710438 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 03:03:45.710830 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.698118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44902 (local address 127.8.208.125:41127)
0929 03:03:45.698323 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:45.698330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:45.698378 (+ 48us) server_negotiation.cc:407] Connection header received
0929 03:03:45.698449 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:45.698453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:45.698510 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:45.698614 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:45.699399 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.700107 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.701311 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:45.701778 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:45.701949 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:45.705458 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:45.705463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:45.705466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:45.705498 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:45.708182 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:45.708831 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:45.708836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:45.708844 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:45.708978 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:45.709408 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:45.709422 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:45.709430 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:45.709701 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:45.709852 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:45.710246 (+ 394us) server_negotiation.cc:299] Negotiation successful
0929 03:03:45.710470 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 03:03:45.712272 31470 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:03:45.713639 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.713079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54040)
0929 03:03:45.713319 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.713431 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 03:03:45.714711 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.714258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54056)
0929 03:03:45.714442 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.714558 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 03:03:45.714954 31470 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:34227: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
I20260929 03:03:45.762765 31544 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } attempt: 3
I20260929 03:03:45.764200 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.763619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54072)
0929 03:03:45.763918 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.764024 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 03:03:45.765444 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.764966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54084)
0929 03:03:45.765237 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.765315 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260929 03:03:45.765688 31544 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:34227: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
I20260929 03:03:45.766213 31470 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } attempt: 1
I20260929 03:03:45.767355 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.766764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54094)
0929 03:03:45.766996 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.767130 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 03:03:45.768420 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.768057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54110)
0929 03:03:45.768213 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.768292 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260929 03:03:45.768674 31470 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:34227: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
I20260929 03:03:45.836900 31470 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } attempt: 2
I20260929 03:03:45.838176 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.837566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54116)
0929 03:03:45.837828 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.837947 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 03:03:45.839133 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.838821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54132)
0929 03:03:45.838958 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.839034 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260929 03:03:45.839442 31470 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:34227: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
W20260929 03:03:45.887153 31551 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 03:03:45.887519 31551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:45.887648 31551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:45.887787 31551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:45.895236 31551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:45.895342 31551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:45.895399 31551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:45.895471 31551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:45.895571 31551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:45.906188 31551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:41205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:34389,127.8.208.125:41127,127.8.208.124:34227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.124.keytab
--principal=kudu/127.8.208.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.124:34227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.208.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:45.907536 31551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:45.908836 31551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:03:45.908970 31544 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } attempt: 4
I20260929 03:03:45.910200 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.909658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54138)
0929 03:03:45.909905 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.910018 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 03:03:45.911211 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.910835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54142)
0929 03:03:45.911003 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.911085 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 03:03:45.911449 31544 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:34227: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
W20260929 03:03:45.921344 31557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:45.921344 31556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:45.921512 31559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:45.922027 31551 server_base.cc:1034] running on GCE node
I20260929 03:03:45.922799 31551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:45.924201 31551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:45.925477 31551 hybrid_clock.cc:648] HybridClock initialized: now 1790651025925385 us; error 56 us; skew 500 ppm
I20260929 03:03:45.925917 31551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:45 dist-test-slave-m1rv krb5kdc[31390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651025, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:45.929302 31551 init.cc:377] Logged in from keytab as kudu/127.8.208.124@KRBTEST.COM (short username kudu)
I20260929 03:03:45.931066 31551 webserver.cc:460] Webserver started at http://127.8.208.124:45003/ using document root <none> and password file <none>
I20260929 03:03:45.931766 31551 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:45.931888 31551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:45.932211 31551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:45.935259 31551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/data/instance:
uuid: "db282fd6047b46ca88d7c5c6f7508b57"
format_stamp: "Formatted at 2026-09-29 03:03:45 on dist-test-slave-m1rv"
server_key: "b59eb6b707d34f2b68cfc43959166faa"
server_key_iv: "c0f62a20671869be3f969c08cf0f3f79"
server_key_version: "encryptionkey@0"
I20260929 03:03:45.936080 31551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/wal/instance:
uuid: "db282fd6047b46ca88d7c5c6f7508b57"
format_stamp: "Formatted at 2026-09-29 03:03:45 on dist-test-slave-m1rv"
server_key: "b59eb6b707d34f2b68cfc43959166faa"
server_key_iv: "c0f62a20671869be3f969c08cf0f3f79"
server_key_version: "encryptionkey@0"
I20260929 03:03:45.941244 31551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 03:03:45.944465 31566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:45.945654 31470 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } attempt: 3
I20260929 03:03:45.945734 31551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:03:45.945899 31551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/wal
uuid: "db282fd6047b46ca88d7c5c6f7508b57"
format_stamp: "Formatted at 2026-09-29 03:03:45 on dist-test-slave-m1rv"
server_key: "b59eb6b707d34f2b68cfc43959166faa"
server_key_iv: "c0f62a20671869be3f969c08cf0f3f79"
server_key_version: "encryptionkey@0"
I20260929 03:03:45.946107 31551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:45.946628 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.946205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54154)
0929 03:03:45.946382 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.946470 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 03:03:45.947563 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:45.947208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54162)
0929 03:03:45.947406 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:45.947464 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260929 03:03:45.947751 31470 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:34227: Network error: Client connection negotiation failed: client connection to 127.8.208.124:34227: connect: Connection refused (error 111)
I20260929 03:03:46.016150 31551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:46.020620 31551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:46.020938 31551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:46.040607 31551 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.124:34227
I20260929 03:03:46.040609 31617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.124:34227 every 8 connection(s)
I20260929 03:03:46.042227 31551 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
I20260929 03:03:46.046109 31618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:46.052070 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 31551
I20260929 03:03:46.052284 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790650672582614-9025-0/minicluster-data/master-2/wal/instance
I20260929 03:03:46.052819 9025 external_mini_cluster.cc:1428] Setting key 9fb49c9d2df9650142e5ee13733c4580
I20260929 03:03:46.052875 31618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:03:46 dist-test-slave-m1rv krb5kdc[31390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651025, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.124@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
Sep 29 03:03:46 dist-test-slave-m1rv krb5kdc[31390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.124@KRBTEST.COM
I20260929 03:03:46.075274 31623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.056011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59678 (local address 127.8.208.126:34389)
0929 03:03:46.056527 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.056533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.057229 (+ 696us) server_negotiation.cc:407] Connection header received
0929 03:03:46.058201 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.058209 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.058285 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.058409 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.060128 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.061382 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.063499 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.064143 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.064279 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.068472 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:46.068478 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:46.068484 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:46.068540 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:46.071595 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.072584 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.072588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.072590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.072718 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.073281 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.073284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.073286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.073647 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:46.073799 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.074777 (+ 978us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.074946 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":245,"threads_started":1}
I20260929 03:03:46.076781 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.055879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:34389 (local address 127.0.0.1:59678)
0929 03:03:46.057047 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:46.057094 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:46.057960 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:46.058474 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:46.058480 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:46.059364 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:46.059902 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:46.059930 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.061553 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.061557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:46.063315 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:46.063343 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.064350 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.064355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:46.064526 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.065403 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:46.065440 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:46.068281 (+ 2841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:46.071739 (+ 3458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:46.071751 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:46.071764 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:46.072459 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:46.072843 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:46.072854 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:46.072858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:46.073106 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:46.073786 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:46.073792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:46.074314 (+ 522us) client_negotiation.cc:769] Sending connection context
0929 03:03:46.074635 (+ 321us) client_negotiation.cc:240] Negotiation successful
0929 03:03:46.075140 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":49,"thread_start_us":684,"threads_started":1}
I20260929 03:03:46.076746 31621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.054774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54168 (local address 127.8.208.124:34227)
0929 03:03:46.055617 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.055631 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.055666 (+ 35us) server_negotiation.cc:407] Connection header received
0929 03:03:46.057947 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.057968 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.058241 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.058451 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.060146 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.061792 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.063502 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.064359 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.064514 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.069522 (+ 5008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:46.069535 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:46.069553 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:46.069604 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:46.073357 (+ 3753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.074193 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.074202 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.074205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.074362 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.074734 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.074739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.074742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.075164 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:46.075492 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.075856 (+ 364us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.076101 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":659,"thread_start_us":405,"threads_started":1}
I20260929 03:03:46.078449 31618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:03:46 dist-test-slave-m1rv krb5kdc[31390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651025, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.124@KRBTEST.COM for kudu/127.8.208.125@KRBTEST.COM
Sep 29 03:03:46 dist-test-slave-m1rv krb5kdc[31390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:03:46.092337 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.079076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:41127 (local address 127.0.0.1:44914)
0929 03:03:46.079237 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:46.079256 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:46.079351 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:46.079881 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:46.079885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:46.080255 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:46.080414 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:46.080424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.081529 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.081537 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:46.082427 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:46.082439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.083208 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.083214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:46.083449 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.084051 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:46.084070 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:46.086223 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:46.089992 (+ 3769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:46.090000 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:46.090004 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:46.090357 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:46.090731 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:46.090736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:46.090740 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:46.090896 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:46.091415 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:46.091420 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:46.091568 (+ 148us) client_negotiation.cc:769] Sending connection context
0929 03:03:46.091706 (+ 138us) client_negotiation.cc:240] Negotiation successful
0929 03:03:46.091981 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 03:03:46.092387 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.079373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44914 (local address 127.8.208.125:41127)
0929 03:03:46.079559 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.079563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.079582 (+ 19us) server_negotiation.cc:407] Connection header received
0929 03:03:46.079636 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.079639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.079739 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.079858 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.080574 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.081380 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.082589 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.083053 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.083206 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.086441 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:46.086446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:46.086450 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:46.086507 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:46.089862 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.090472 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.090479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.090483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.090620 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.091004 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.091011 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.091014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.091280 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:46.091451 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.091800 (+ 349us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.092003 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:03:46.093632 31618 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:03:46.093804 31623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.079875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59688 (local address 127.8.208.126:34389)
0929 03:03:46.080021 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.080028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.080079 (+ 51us) server_negotiation.cc:407] Connection header received
0929 03:03:46.080249 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.080253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.080332 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.080430 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.081555 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.082273 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.083448 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.083853 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.083988 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.088172 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:46.088176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:46.088179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:46.088207 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:46.090746 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.091372 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.091378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.091386 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.091495 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.091854 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.091860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.091864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.092130 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:46.092394 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.093241 (+ 847us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.093472 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 03:03:46 dist-test-slave-m1rv krb5kdc[31390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651025, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.124@KRBTEST.COM for kudu/127.8.208.124@KRBTEST.COM
Sep 29 03:03:46 dist-test-slave-m1rv krb5kdc[31390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.125@KRBTEST.COM
I20260929 03:03:46.108136 31621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.094580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54172 (local address 127.8.208.124:34227)
0929 03:03:46.094723 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.094728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.094748 (+ 20us) server_negotiation.cc:407] Connection header received
0929 03:03:46.094805 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.094809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.094865 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.094953 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.095945 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.097064 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.098743 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.099377 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.099510 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.102646 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:46.102650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:46.102653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:46.102679 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:46.105313 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.106086 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.106090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.106092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.106218 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.106676 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.106680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.106682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.106943 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:46.107056 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.107504 (+ 448us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.107846 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":183,"server-negotiator.queue_time_us":51}
I20260929 03:03:46.108160 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.094341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54172)
0929 03:03:46.094484 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:46.094509 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:46.094598 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:46.095073 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:46.095081 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:46.095626 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:46.095815 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:46.095830 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.097280 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.097283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:46.098595 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:46.098609 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.099521 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.099525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:46.099721 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.100280 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:46.100296 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:46.102460 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:46.105441 (+ 2981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:46.105445 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:46.105447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:46.105972 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:46.106347 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:46.106351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:46.106353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:46.106567 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:46.107069 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:46.107075 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:46.107230 (+ 155us) client_negotiation.cc:769] Sending connection context
0929 03:03:46.107394 (+ 164us) client_negotiation.cc:240] Negotiation successful
0929 03:03:46.107715 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":21}
I20260929 03:03:46.110801 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.095569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44922 (local address 127.8.208.125:41127)
0929 03:03:46.095777 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.095784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.095804 (+ 20us) server_negotiation.cc:407] Connection header received
0929 03:03:46.095867 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.095876 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.095958 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.096088 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.097558 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.098612 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.100098 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.100511 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.100649 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.104719 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:46.104724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:46.104727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:46.104757 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:46.108014 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.108747 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.108756 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.108762 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.108923 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.109405 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.109412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.109437 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.109787 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:46.109985 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.110342 (+ 357us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.110535 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 03:03:46.114253 31618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57: Bootstrap starting.
I20260929 03:03:46.115779 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:46.117897 31618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:46.119263 31618 log.cc:824] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:46.118539 31445 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2c4dc94a3f9944bcb2556b937b581ad4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59688
I20260929 03:03:46.122457 31618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57: No bootstrap required, opened a new log
I20260929 03:03:46.124951 31470 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } attempt: 4
Sep 29 03:03:46 dist-test-slave-m1rv krb5kdc[31390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651024, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for kudu/127.8.208.124@KRBTEST.COM
I20260929 03:03:46.133671 31618 raft_consensus.cc:348] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } }
I20260929 03:03:46.134035 31618 raft_consensus.cc:729] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db282fd6047b46ca88d7c5c6f7508b57, State: Initialized, Role: FOLLOWER
I20260929 03:03:46.134617 31618 consensus_queue.cc:260] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } }
I20260929 03:03:46.136111 31625 sys_catalog.cc:455] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } } }
I20260929 03:03:46.136606 31625 sys_catalog.cc:458] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:46.137069 31618 sys_catalog.cc:564] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:46.140465 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.125603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54178)
0929 03:03:46.125783 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:46.125803 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:46.125925 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:46.126386 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:46.126391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:46.126798 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:46.126952 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:46.126961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.128020 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.128028 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:46.128955 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:46.128966 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.129959 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.129965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:46.130172 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.130822 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:46.130839 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:46.133294 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:46.137564 (+ 4270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:46.137570 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:46.137574 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:46.137964 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:46.138393 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:46.138399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:46.138405 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:46.138596 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:46.139427 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:46.139433 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:46.139623 (+ 190us) client_negotiation.cc:769] Sending connection context
0929 03:03:46.139786 (+ 163us) client_negotiation.cc:240] Negotiation successful
0929 03:03:46.140069 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260929 03:03:46.140465 31621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.125768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54178 (local address 127.8.208.124:34227)
0929 03:03:46.126036 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.126042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.126063 (+ 21us) server_negotiation.cc:407] Connection header received
0929 03:03:46.126116 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.126119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.126189 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.126325 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.127105 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.127861 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.129095 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.129750 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.129942 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.133511 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:46.133518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:46.133523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:46.133562 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:46.137425 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.138085 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.138092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.138095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.138213 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.138731 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.138734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.138736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.139212 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:46.139460 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.139881 (+ 421us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.140090 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 03:03:46.146385 31470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4: Bootstrap starting.
W20260929 03:03:46.148131 31636 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:03:46.148305 31636 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:03:46.150311 31470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:46.151628 31470 log.cc:824] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:46.155259 31470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4: No bootstrap required, opened a new log
I20260929 03:03:46.145495 31551 master_runner.cc:186] Error getting master registration for 127.8.208.126:34389: OK, instance_id { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" instance_seqno: 1790651024987410 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 03:03:46.156898 31551 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 03:03:46.162848 31470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } }
I20260929 03:03:46.163251 31470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c4dc94a3f9944bcb2556b937b581ad4, State: Initialized, Role: FOLLOWER
I20260929 03:03:46.163769 31470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } }
I20260929 03:03:46.164747 31639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } } }
I20260929 03:03:46.165133 31639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:46.166074 31470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:46.166257 31445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 03:03:46.166471 31445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:46.169322 31445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } }
I20260929 03:03:46.171608 31593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "db282fd6047b46ca88d7c5c6f7508b57"
I20260929 03:03:46.172086 31593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:46.173311 31445 leader_election.cc:288] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [CANDIDATE]: Term 1 election: Requested vote from peers 47e1f269c5244196b09e6e4c4663a02b (127.8.208.125:41127), db282fd6047b46ca88d7c5c6f7508b57 (127.8.208.124:34227)
I20260929 03:03:46.175575 31593 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c4dc94a3f9944bcb2556b937b581ad4 in term 1.
I20260929 03:03:46.176367 31422 leader_election.cc:302] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c4dc94a3f9944bcb2556b937b581ad4, db282fd6047b46ca88d7c5c6f7508b57; no voters:
I20260929 03:03:46.176901 31639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:46.188350 31639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [term 1 LEADER]: Becoming Leader. State: Replica: 2c4dc94a3f9944bcb2556b937b581ad4, State: Running, Role: LEADER
I20260929 03:03:46.189191 31639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } }
I20260929 03:03:46.197914 31647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c4dc94a3f9944bcb2556b937b581ad4. Latest consensus state: current_term: 1 leader_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } } }
I20260929 03:03:46.198518 31647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:46.202929 31654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:46.210642 31544 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } attempt: 5
I20260929 03:03:46.217206 31654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 03:03:46 dist-test-slave-m1rv krb5kdc[31390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651025, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.125@KRBTEST.COM for kudu/127.8.208.124@KRBTEST.COM
I20260929 03:03:46.213335 31403 master_runner.cc:186] Error getting master registration for 127.8.208.125:41127: OK, instance_id { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" instance_seqno: 1790651025483950 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 03:03:46.225327 31403 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 03:03:46.226783 31424 tablet.cc:2369] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:03:46.231194 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.211339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:34227 (local address 127.0.0.1:54182)
0929 03:03:46.211706 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:46.211729 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:46.211842 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:46.212723 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:46.212728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:46.213309 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:46.213524 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:46.213542 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.214977 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.214983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:46.216420 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:46.216438 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.217749 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.217755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:46.218027 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.218696 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:46.218719 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:46.222267 (+ 3548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:46.227681 (+ 5414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:46.227690 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:46.227697 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:46.228367 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:46.228900 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:46.228908 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:46.228914 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:46.229117 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:46.229901 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:46.229909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:46.230124 (+ 215us) client_negotiation.cc:769] Sending connection context
0929 03:03:46.230278 (+ 154us) client_negotiation.cc:240] Negotiation successful
0929 03:03:46.230644 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240}
I20260929 03:03:46.232547 31621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.212067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54182 (local address 127.8.208.124:34227)
0929 03:03:46.212231 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.212239 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.212259 (+ 20us) server_negotiation.cc:407] Connection header received
0929 03:03:46.212323 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.212328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.212426 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.212535 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.213713 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.214799 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.216630 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.217391 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.217526 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.222498 (+ 4972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:46.222506 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:46.222512 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:46.222552 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:46.227494 (+ 4942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.228556 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.228563 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.228569 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.228760 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.229309 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.229317 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.229322 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.229741 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:46.231573 (+ 1832us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.231719 (+ 146us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.232034 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:03:46.238792 31544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b: Bootstrap starting.
I20260929 03:03:46.243412 31544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:46.244324 31593 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [term 1 FOLLOWER]: Refusing update from remote peer 2c4dc94a3f9944bcb2556b937b581ad4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:03:46.244928 31544 log.cc:824] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:46.245458 31647 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:03:46.263921 31623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.220063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59704 (local address 127.8.208.126:34389)
0929 03:03:46.220313 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.220326 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.234207 (+ 13881us) server_negotiation.cc:407] Connection header received
0929 03:03:46.234437 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.234443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.234535 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.234634 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.237820 (+ 3186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.239003 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.249290 (+ 10287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.249924 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.250031 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.256066 (+ 6035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:46.256075 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:46.256082 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:46.256163 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:46.260156 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.261010 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.261017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.261022 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.261163 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.261667 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.261671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.261674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.262071 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:46.262338 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.262908 (+ 570us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.263450 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 03:03:46.267798 31544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b: No bootstrap required, opened a new log
I20260929 03:03:46.275022 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.242716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44938 (local address 127.8.208.125:41127)
0929 03:03:46.242870 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.242875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.251285 (+ 8410us) server_negotiation.cc:407] Connection header received
0929 03:03:46.251773 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.251780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.251870 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.251978 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.257646 (+ 5668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.258839 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.261793 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.262434 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.262576 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.266421 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:46.266427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:46.266432 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:46.266469 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:46.270337 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.271266 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.271275 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.271279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.271577 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.272109 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.272114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.272118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.272759 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:46.273343 (+ 584us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.274356 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.274622 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260929 03:03:46.277563 31572 tablet.cc:2369] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:03:46.286015 31544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } }
I20260929 03:03:46.286537 31544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47e1f269c5244196b09e6e4c4663a02b, State: Initialized, Role: FOLLOWER
I20260929 03:03:46.287180 31544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } }
I20260929 03:03:46.290920 31625 sys_catalog.cc:455] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c4dc94a3f9944bcb2556b937b581ad4. Latest consensus state: current_term: 1 leader_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } } }
I20260929 03:03:46.291232 31625 sys_catalog.cc:458] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:46.295418 31662 mvcc.cc:205] Tried to move back new op lower bound from 7334506603481837568 to 7334506603278888960. Current Snapshot: MvccSnapshot[applied={T|T < 7334506603481837568}]
I20260929 03:03:46.297519 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.223564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54198 (local address 127.8.208.124:34227)
0929 03:03:46.233607 (+ 10043us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.233613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.242050 (+ 8437us) server_negotiation.cc:407] Connection header received
0929 03:03:46.242217 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.242222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.242304 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.242414 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.277363 (+ 34949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.278767 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.280838 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.281761 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.281917 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.285680 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:46.285688 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:46.285693 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:46.285739 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:46.290421 (+ 4682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.291971 (+ 1550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.292013 (+ 42us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.292020 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.292352 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:46.293147 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:46.293151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:46.293201 (+ 50us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:46.293829 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:46.294262 (+ 433us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.295178 (+ 916us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.295662 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9916,"thread_start_us":9706,"threads_started":1}
I20260929 03:03:46.297907 31625 sys_catalog.cc:455] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } } }
I20260929 03:03:46.298242 31625 sys_catalog.cc:458] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:46.302811 31639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } } }
I20260929 03:03:46.303277 31639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:46.307282 31667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } } }
I20260929 03:03:46.308744 31544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:46.308820 31667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:46.308796 31519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "47e1f269c5244196b09e6e4c4663a02b"
I20260929 03:03:46.309145 31518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:46.314262 31518 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [term 1 FOLLOWER]: Refusing update from remote peer 2c4dc94a3f9944bcb2556b937b581ad4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:03:46.315716 31654 catalog_manager.cc:1347] Generated new cluster ID: 763a8cec74354db196182c928e13948b
I20260929 03:03:46.315814 31654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:46.317409 31639 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:03:46.320236 31665 mvcc.cc:205] Tried to move back new op lower bound from 7334506603481837568 to 7334506603278888960. Current Snapshot: MvccSnapshot[applied={T|T < 7334506603481837568 or (T in {7334506603481837568})}]
I20260929 03:03:46.341984 31654 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:46.344327 31654 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:46.362156 31654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4: Generated new TSK 0
I20260929 03:03:46.363952 31654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:46.369416 31519 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2c4dc94a3f9944bcb2556b937b581ad4 for term 1 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260929 03:03:46.370102 31679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } } }
I20260929 03:03:46.370527 31679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:46.370649 31670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } } }
I20260929 03:03:46.370886 31667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c4dc94a3f9944bcb2556b937b581ad4. Latest consensus state: current_term: 1 leader_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c4dc94a3f9944bcb2556b937b581ad4" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 34389 } } peers { permanent_uuid: "47e1f269c5244196b09e6e4c4663a02b" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 41127 } } peers { permanent_uuid: "db282fd6047b46ca88d7c5c6f7508b57" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 34227 } } }
I20260929 03:03:46.371086 31670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c4dc94a3f9944bcb2556b937b581ad4 [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:46.371176 31667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:46.396386 31685 catalog_manager.cc:1259] Loaded cluster ID: 763a8cec74354db196182c928e13948b
I20260929 03:03:46.396533 31685 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b: loading cluster ID for follower catalog manager: success
I20260929 03:03:46.400264 31685 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b: acquiring CA information for follower catalog manager: success
I20260929 03:03:46.402928 31685 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 47e1f269c5244196b09e6e4c4663a02b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 03:03:46.498728 31660 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:46.489265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54208 (local address 127.8.208.124:34227)
0929 03:03:46.489527 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.489534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.495069 (+ 5535us) server_negotiation.cc:407] Connection header received
0929 03:03:46.496695 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.496701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.496796 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.496889 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.498460 (+ 1571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54208: BlockingRecv error: recv got EOF from 127.0.0.1:54208 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260929 03:03:46.500160 31623 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:46.480826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59710 (local address 127.8.208.126:34389)
0929 03:03:46.481058 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.481065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.494588 (+ 13523us) server_negotiation.cc:407] Connection header received
0929 03:03:46.497143 (+ 2555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.497149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.497250 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.497328 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.499929 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59710: BlockingRecv error: recv got EOF from 127.0.0.1:59710 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260929 03:03:46.514328 31548 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:46.495781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44950 (local address 127.8.208.125:41127)
0929 03:03:46.496014 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.496020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.501470 (+ 5450us) server_negotiation.cc:407] Connection header received
0929 03:03:46.501618 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.501623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.501716 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.501822 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.514102 (+ 12280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44950: BlockingRecv error: recv got EOF from 127.0.0.1:44950 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 03:03:46.564148 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.546649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44954 (local address 127.8.208.125:41127)
0929 03:03:46.546846 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.546853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.556069 (+ 9216us) server_negotiation.cc:407] Connection header received
0929 03:03:46.556713 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.556720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.556959 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.557110 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:46.557887 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.559037 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.561102 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.561815 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.561913 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.562662 (+ 749us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:46.563347 (+ 685us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:46.563474 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.563719 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.563816 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260929 03:03:46.609236 31660 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:46.599671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.8.208.124:34227)
0929 03:03:46.599864 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.599870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.606695 (+ 6825us) server_negotiation.cc:407] Connection header received
0929 03:03:46.607216 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.607225 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.607372 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.607453 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.608954 (+ 1501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54218: BlockingRecv error: recv got EOF from 127.0.0.1:54218 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 03:03:46.661660 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.643611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44970 (local address 127.8.208.125:41127)
0929 03:03:46.643824 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.643830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.654613 (+ 10783us) server_negotiation.cc:407] Connection header received
0929 03:03:46.655156 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.655162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.655254 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.655317 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:46.656130 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.657341 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.659370 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.660114 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.660272 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.660772 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:46.660944 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:46.661005 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.661257 (+ 252us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.661356 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260929 03:03:46.706743 31660 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:46.697676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54222 (local address 127.8.208.124:34227)
0929 03:03:46.697853 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.697859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.704702 (+ 6843us) server_negotiation.cc:407] Connection header received
0929 03:03:46.705246 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.705252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.705355 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.705439 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.706502 (+ 1063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54222: BlockingRecv error: recv got EOF from 127.0.0.1:54222 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 03:03:46.764493 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.746112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44986 (local address 127.8.208.125:41127)
0929 03:03:46.746315 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.746322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.757252 (+ 10930us) server_negotiation.cc:407] Connection header received
0929 03:03:46.757816 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.757823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.757925 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.758000 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:46.758708 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.759924 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.762013 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.762737 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.762849 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.763553 (+ 704us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:46.763726 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:46.763795 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.764026 (+ 231us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.764117 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260929 03:03:46.811640 31660 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:46.797403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54234 (local address 127.8.208.124:34227)
0929 03:03:46.797627 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.797633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.809768 (+ 12135us) server_negotiation.cc:407] Connection header received
0929 03:03:46.810019 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.810025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.810150 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.810245 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.811364 (+ 1119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54234: BlockingRecv error: recv got EOF from 127.0.0.1:54234 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 03:03:46.872697 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.853374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44996 (local address 127.8.208.125:41127)
0929 03:03:46.853591 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.853597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.864678 (+ 11081us) server_negotiation.cc:407] Connection header received
0929 03:03:46.865232 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.865238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.865333 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.865443 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:46.866150 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.867403 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.869540 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.870294 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.870421 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.871409 (+ 988us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:46.871551 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:46.871622 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.872304 (+ 682us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.872407 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260929 03:03:46.885962 31660 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:46.883388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54240 (local address 127.8.208.124:34227)
0929 03:03:46.883635 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.883642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.884235 (+ 593us) server_negotiation.cc:407] Connection header received
0929 03:03:46.884466 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.884472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.884572 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.884691 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.885695 (+ 1004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54240: BlockingRecv error: recv got EOF from 127.0.0.1:54240 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
I20260929 03:03:46.906407 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.897756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45004 (local address 127.8.208.125:41127)
0929 03:03:46.897978 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.897985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.898584 (+ 599us) server_negotiation.cc:407] Connection header received
0929 03:03:46.898824 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.898830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.898949 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.899066 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:46.899610 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.900816 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.904079 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.904765 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.904912 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.905510 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:46.905663 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:46.905771 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.905953 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.906048 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260929 03:03:46.916662 31660 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:46.914217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54246 (local address 127.8.208.124:34227)
0929 03:03:46.914430 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.914437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.914998 (+ 561us) server_negotiation.cc:407] Connection header received
0929 03:03:46.915237 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.915243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.915341 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.915451 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.916396 (+ 945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54246: BlockingRecv error: recv got EOF from 127.0.0.1:54246 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 03:03:46.944370 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.936668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45012 (local address 127.8.208.125:41127)
0929 03:03:46.936894 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.936901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.937524 (+ 623us) server_negotiation.cc:407] Connection header received
0929 03:03:46.937850 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.937857 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.937951 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.938057 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:46.938644 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.939842 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.942030 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.942788 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.942928 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.943467 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:46.943619 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:46.943708 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.943941 (+ 233us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.944032 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260929 03:03:46.957448 31660 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:46.954850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54258 (local address 127.8.208.124:34227)
0929 03:03:46.955028 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.955034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.955660 (+ 626us) server_negotiation.cc:407] Connection header received
0929 03:03:46.955883 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.955890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.956029 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.956197 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:46.957143 (+ 946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54258: BlockingRecv error: recv got EOF from 127.0.0.1:54258 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 03:03:46.998395 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:46.992596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.8.208.125:41127)
0929 03:03:46.992802 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:46.992809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:46.993212 (+ 403us) server_negotiation.cc:407] Connection header received
0929 03:03:46.993425 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:46.993431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:46.993543 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:46.993652 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:46.994063 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.995156 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.996433 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:46.997211 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:46.997330 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:46.997681 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:46.997810 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:46.997893 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:46.998023 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 03:03:46.998101 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260929 03:03:47.007051 31660 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:47.005027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54270 (local address 127.8.208.124:34227)
0929 03:03:47.005297 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:47.005303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:47.005612 (+ 309us) server_negotiation.cc:407] Connection header received
0929 03:03:47.005793 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:47.005797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:47.005877 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:47.005969 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:47.006790 (+ 821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54270: BlockingRecv error: recv got EOF from 127.0.0.1:54270 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 03:03:47.079272 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:47.074403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.8.208.125:41127)
0929 03:03:47.074592 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:47.074597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:47.075000 (+ 403us) server_negotiation.cc:407] Connection header received
0929 03:03:47.075153 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:47.075157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:47.075229 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:47.075329 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:47.075700 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:47.076482 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:47.077834 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:47.078252 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:47.078354 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:47.078704 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:47.078802 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:47.078871 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:47.079002 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 03:03:47.079067 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260929 03:03:47.091292 31660 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:47.089336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54282 (local address 127.8.208.124:34227)
0929 03:03:47.089597 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:47.089606 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:47.089890 (+ 284us) server_negotiation.cc:407] Connection header received
0929 03:03:47.090140 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:47.090145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:47.090215 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:47.090346 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:47.091113 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54282: BlockingRecv error: recv got EOF from 127.0.0.1:54282 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260929 03:03:47.150767 31636 catalog_manager.cc:1259] Loaded cluster ID: 763a8cec74354db196182c928e13948b
I20260929 03:03:47.150902 31636 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57: loading cluster ID for follower catalog manager: success
I20260929 03:03:47.153609 31636 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57: acquiring CA information for follower catalog manager: success
I20260929 03:03:47.155495 31636 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db282fd6047b46ca88d7c5c6f7508b57: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 03:03:47.231840 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:47.226914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.8.208.125:41127)
0929 03:03:47.227099 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:47.227104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:47.227518 (+ 414us) server_negotiation.cc:407] Connection header received
0929 03:03:47.227681 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:47.227685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:47.227769 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:47.227860 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:47.228295 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:47.228988 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:47.230384 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:47.230818 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:47.230909 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:47.231273 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:47.231370 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:47.231435 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:47.231558 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 03:03:47.231623 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:03:47.242969 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:47.237472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54296 (local address 127.8.208.124:34227)
0929 03:03:47.237640 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:47.237645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:47.238045 (+ 400us) server_negotiation.cc:407] Connection header received
0929 03:03:47.238216 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:47.238221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:47.238384 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:47.238477 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:47.238841 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:47.239572 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:47.240979 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:47.241423 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:47.241521 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:47.241966 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:47.242459 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:47.242589 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:47.242675 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 03:03:47.242737 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:03:47.257105 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:47.249307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54308 (local address 127.8.208.124:34227)
0929 03:03:47.249499 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:47.249504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:47.250608 (+ 1104us) server_negotiation.cc:407] Connection header received
0929 03:03:47.250802 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:47.250807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:47.250880 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:47.250936 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:47.251718 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:47.252579 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:47.254765 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:47.255216 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:47.255353 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:47.256301 (+ 948us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:47.256437 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:47.256529 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:47.256739 (+ 210us) server_negotiation.cc:299] Negotiation successful
0929 03:03:47.256825 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 03:03:47.257383 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:47.249190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59714 (local address 127.8.208.126:34389)
0929 03:03:47.249892 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:47.249901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:47.250341 (+ 440us) server_negotiation.cc:407] Connection header received
0929 03:03:47.250889 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:47.250894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:47.251120 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:47.251304 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:47.251835 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:47.252993 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:47.254838 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:47.255518 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:47.255689 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:47.256171 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:47.256685 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:47.256801 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:47.256876 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 03:03:47.256959 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":404,"threads_started":1}
I20260929 03:03:47.257854 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:47.250540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45038 (local address 127.8.208.125:41127)
0929 03:03:47.250753 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:47.250760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:47.251145 (+ 385us) server_negotiation.cc:407] Connection header received
0929 03:03:47.251812 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:47.251817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:47.251897 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:47.252002 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:47.252551 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:47.253430 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:47.255514 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:47.256176 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:47.256298 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:47.256896 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:47.257024 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:47.257093 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:47.257521 (+ 428us) server_negotiation.cc:299] Negotiation successful
0929 03:03:47.257602 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 03:03:47.263025 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 31403
I20260929 03:03:47.264040 31465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:47.373688 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 31403
W20260929 03:03:47.393787 31567 connection.cc:535] server connection from 127.0.0.1:54178 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 03:03:47.394515 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 31477
I20260929 03:03:47.395601 31539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:47.503103 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 31477
I20260929 03:03:47.522931 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 31551
I20260929 03:03:47.524075 31613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:47.631220 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 31551
2026-09-29T03:03:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4482 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:03:47 dist-test-slave-m1rv krb5kdc[31800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:03:47 dist-test-slave-m1rv krb5kdc[31800](info): set up 2 sockets
Sep 29 03:03:47 dist-test-slave-m1rv krb5kdc[31800](info): commencing operation
krb5kdc: starting...
W20260929 03:03:48.745909 9025 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/test-user.keytab.
Sep 29 03:03:48 dist-test-slave-m1rv krb5kdc[31800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:03:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:03:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.208.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
Entry for principal kudu/127.8.208.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab.
I20260929 03:03:49.005267 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:43567
--webserver_interface=127.8.208.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:36555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:43567,127.8.208.125:35633,127.8.208.124:38379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:49.329421 31813 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 03:03:49.329800 31813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:49.329933 31813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:49.330076 31813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:49.337433 31813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:49.337580 31813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:49.337690 31813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:49.337780 31813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:49.337872 31813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:49.347621 31813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:36555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:43567,127.8.208.125:35633,127.8.208.124:38379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.126.keytab
--principal=kudu/127.8.208.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.126:43567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.208.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:49.348927 31813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:49.350247 31813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:49.362130 31818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:49.362166 31819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:49.362228 31821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:49.362895 31813 server_base.cc:1034] running on GCE node
I20260929 03:03:49.363664 31813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:49.365123 31813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:49.366384 31813 hybrid_clock.cc:648] HybridClock initialized: now 1790651029366307 us; error 53 us; skew 500 ppm
I20260929 03:03:49.366724 31813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:49 dist-test-slave-m1rv krb5kdc[31800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651029, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:49.369972 31813 init.cc:377] Logged in from keytab as kudu/127.8.208.126@KRBTEST.COM (short username kudu)
I20260929 03:03:49.371518 31813 webserver.cc:460] Webserver started at http://127.8.208.126:39677/ using document root <none> and password file <none>
I20260929 03:03:49.372210 31813 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:49.372339 31813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:49.372651 31813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:49.375567 31813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/data/instance:
uuid: "311486e41c304d1cb8329201a2b9fa9b"
format_stamp: "Formatted at 2026-09-29 03:03:49 on dist-test-slave-m1rv"
server_key: "ae99782d8db2d5004d565423a8fc63c0"
server_key_iv: "5fb4528096f6bf6b15d12e01bdac2452"
server_key_version: "encryptionkey@0"
I20260929 03:03:49.376320 31813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/wal/instance:
uuid: "311486e41c304d1cb8329201a2b9fa9b"
format_stamp: "Formatted at 2026-09-29 03:03:49 on dist-test-slave-m1rv"
server_key: "ae99782d8db2d5004d565423a8fc63c0"
server_key_iv: "5fb4528096f6bf6b15d12e01bdac2452"
server_key_version: "encryptionkey@0"
I20260929 03:03:49.381316 31813 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260929 03:03:49.384687 31828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:49.385974 31813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:03:49.386183 31813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/wal
uuid: "311486e41c304d1cb8329201a2b9fa9b"
format_stamp: "Formatted at 2026-09-29 03:03:49 on dist-test-slave-m1rv"
server_key: "ae99782d8db2d5004d565423a8fc63c0"
server_key_iv: "5fb4528096f6bf6b15d12e01bdac2452"
server_key_version: "encryptionkey@0"
I20260929 03:03:49.386407 31813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:49.399893 31813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:49.404249 31813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:49.404536 31813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:49.422806 31813 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.126:43567
I20260929 03:03:49.422814 31879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.126:43567 every 8 connection(s)
I20260929 03:03:49.424134 31813 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/data/info.pb
I20260929 03:03:49.427942 31880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:49.429673 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 31813
I20260929 03:03:49.429869 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-0/wal/instance
I20260929 03:03:49.430305 9025 external_mini_cluster.cc:1428] Setting key 84b35207a798ff2a677c7e0982d649ea
I20260929 03:03:49.434530 31880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:03:49 dist-test-slave-m1rv krb5kdc[31800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651029, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.208.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.125@KRBTEST.COM" created.
I20260929 03:03:49.459357 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.437707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:43567 (local address 127.0.0.1:37054)
0929 03:03:49.438495 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.438546 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:49.441414 (+ 2868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:49.442265 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:49.442275 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:49.443060 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:49.443564 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:49.443589 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:49.445137 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:49.445145 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:49.447076 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:49.447092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:49.448013 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:49.448023 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:49.448266 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:49.448975 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:49.449008 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:49.452109 (+ 3101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:49.455279 (+ 3170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:49.455294 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:49.455308 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:49.455844 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:49.456259 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:49.456263 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:49.456265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:49.456404 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:49.457224 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:49.457234 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:49.457419 (+ 185us) client_negotiation.cc:769] Sending connection context
0929 03:03:49.457622 (+ 203us) client_negotiation.cc:240] Negotiation successful
0929 03:03:49.458199 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":155,"thread_start_us":332,"threads_started":1}
I20260929 03:03:49.459447 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.438945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.8.208.126:43567)
0929 03:03:49.439556 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:49.439562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:49.439605 (+ 43us) server_negotiation.cc:407] Connection header received
0929 03:03:49.441776 (+ 2171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:49.441781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:49.442022 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:49.442182 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:49.443796 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:49.444966 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:49.447231 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:49.447767 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:49.447897 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:49.452316 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:49.452324 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:49.452336 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:49.452368 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:49.455144 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:49.455990 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:49.455999 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:49.456004 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:49.456152 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:49.456514 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:49.456517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:49.456519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:49.457068 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:49.457276 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:49.457800 (+ 524us) server_negotiation.cc:299] Negotiation successful
0929 03:03:49.458075 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":411,"thread_start_us":266,"threads_started":1}
I20260929 03:03:49.462548 31880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:03:49.463635 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.463245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33300)
0929 03:03:49.463424 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.463530 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260929 03:03:49.463968 31829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
I20260929 03:03:49.465518 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.465140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33316)
0929 03:03:49.465336 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.465396 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260929 03:03:49.465727 31880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:35633: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.125.keytab.
Entry for principal kudu/127.8.208.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.125.keytab.
I20260929 03:03:49.489364 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.125:35633
--webserver_interface=127.8.208.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.125.keytab
--principal=kudu/127.8.208.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:36555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:43567,127.8.208.125:35633,127.8.208.124:38379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:03:49.515095 31880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } attempt: 1
I20260929 03:03:49.516556 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.515919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33318)
0929 03:03:49.516170 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.516355 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260929 03:03:49.517767 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.517322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33326)
0929 03:03:49.517496 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.517593 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 03:03:49.518013 31880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:35633: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
I20260929 03:03:49.586311 31880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } attempt: 2
I20260929 03:03:49.587770 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.587088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33328)
0929 03:03:49.587409 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.587537 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260929 03:03:49.589068 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.588523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33342)
0929 03:03:49.588714 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.588858 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 03:03:49.589335 31880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:35633: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
I20260929 03:03:49.680683 31880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } attempt: 3
I20260929 03:03:49.682194 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.681501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33352)
0929 03:03:49.681827 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.681976 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260929 03:03:49.683544 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.683008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33356)
0929 03:03:49.683209 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.683321 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260929 03:03:49.683799 31880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:35633: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
I20260929 03:03:49.827075 31880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } attempt: 4
I20260929 03:03:49.828280 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.827768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33364)
0929 03:03:49.828070 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.828153 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260929 03:03:49.828801 31887 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 03:03:49.829267 31887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:49.829406 31887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260929 03:03:49.829449 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.829016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33372)
0929 03:03:49.829194 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.829289 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260929 03:03:49.829572 31887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:49.829696 31880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.125:35633: Network error: Client connection negotiation failed: client connection to 127.8.208.125:35633: connect: Connection refused (error 111)
W20260929 03:03:49.837023 31887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:49.837152 31887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:49.837301 31887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:49.837400 31887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:49.837496 31887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:49.847342 31887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:36555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:43567,127.8.208.125:35633,127.8.208.124:38379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.125.keytab
--principal=kudu/127.8.208.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.125:35633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.208.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:49.848654 31887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:49.850077 31887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:03:49.862373 31892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:49.862349 31895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:49.862540 31893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:49.863257 31887 server_base.cc:1034] running on GCE node
I20260929 03:03:49.864117 31887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:49.865510 31887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:49.866801 31887 hybrid_clock.cc:648] HybridClock initialized: now 1790651029866731 us; error 57 us; skew 500 ppm
I20260929 03:03:49.867225 31887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:49 dist-test-slave-m1rv krb5kdc[31800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651029, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:49.870539 31887 init.cc:377] Logged in from keytab as kudu/127.8.208.125@KRBTEST.COM (short username kudu)
I20260929 03:03:49.872244 31887 webserver.cc:460] Webserver started at http://127.8.208.125:40653/ using document root <none> and password file <none>
I20260929 03:03:49.873294 31887 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:49.873451 31887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:49.873881 31887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:49.878142 31887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/data/instance:
uuid: "6049a9d348934151ba369400ebd5428f"
format_stamp: "Formatted at 2026-09-29 03:03:49 on dist-test-slave-m1rv"
server_key: "b2bd5fa8353d5f08404bfaf703a08904"
server_key_iv: "5fab29293eb60c6c3bdd3c32223086b4"
server_key_version: "encryptionkey@0"
I20260929 03:03:49.878914 31887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/wal/instance:
uuid: "6049a9d348934151ba369400ebd5428f"
format_stamp: "Formatted at 2026-09-29 03:03:49 on dist-test-slave-m1rv"
server_key: "b2bd5fa8353d5f08404bfaf703a08904"
server_key_iv: "5fab29293eb60c6c3bdd3c32223086b4"
server_key_version: "encryptionkey@0"
I20260929 03:03:49.884006 31887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 03:03:49.887281 31902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:49.888664 31887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:03:49.888871 31887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/wal
uuid: "6049a9d348934151ba369400ebd5428f"
format_stamp: "Formatted at 2026-09-29 03:03:49 on dist-test-slave-m1rv"
server_key: "b2bd5fa8353d5f08404bfaf703a08904"
server_key_iv: "5fab29293eb60c6c3bdd3c32223086b4"
server_key_version: "encryptionkey@0"
I20260929 03:03:49.889117 31887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:49.903578 31887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:49.907936 31887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:49.908233 31887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:49.926327 31887 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.125:35633
I20260929 03:03:49.926352 31953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.125:35633 every 8 connection(s)
I20260929 03:03:49.927645 31887 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/data/info.pb
I20260929 03:03:49.931200 31954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:49.933533 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 31887
I20260929 03:03:49.933734 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-1/wal/instance
I20260929 03:03:49.934262 9025 external_mini_cluster.cc:1428] Setting key 989775821f1775226a61d0dd298aa32e
I20260929 03:03:49.940402 31954 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.208.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.208.124@KRBTEST.COM" created.
Sep 29 03:03:49 dist-test-slave-m1rv krb5kdc[31800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651029, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.125@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:03:49.972761 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.943745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37056 (local address 127.8.208.126:43567)
0929 03:03:49.943987 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:49.943995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:49.945306 (+ 1311us) server_negotiation.cc:407] Connection header received
0929 03:03:49.948223 (+ 2917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:49.948227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:49.948301 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:49.948392 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:49.950458 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:49.951130 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:49.953535 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:49.953934 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:49.954045 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:49.966252 (+ 12207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:49.966257 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:49.966260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:49.966296 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:49.969068 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:49.970133 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:49.970140 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:49.970146 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:49.970283 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:49.970783 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:49.970790 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:49.970795 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:49.971153 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:49.971284 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:49.972285 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0929 03:03:49.972507 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 03:03:49.974205 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.944207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:43567 (local address 127.0.0.1:37056)
0929 03:03:49.945106 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.945155 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:49.947901 (+ 2746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:49.948663 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:49.948677 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:49.949496 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:49.950280 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:49.950307 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:49.951290 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:49.951299 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:49.953383 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:49.953400 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:49.954170 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:49.954179 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:49.954495 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:49.961863 (+ 7368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:49.961910 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:49.965944 (+ 4034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:49.969258 (+ 3314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:49.969271 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:49.969291 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:49.969961 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:49.970404 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:49.970411 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:49.970417 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:49.970648 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:49.971295 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:49.971306 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:49.971834 (+ 528us) client_negotiation.cc:769] Sending connection context
0929 03:03:49.972166 (+ 332us) client_negotiation.cc:240] Negotiation successful
0929 03:03:49.972750 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":389,"threads_started":1}
I20260929 03:03:49.976003 31954 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:03:49 dist-test-slave-m1rv krb5kdc[31800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651029, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.125@KRBTEST.COM for kudu/127.8.208.125@KRBTEST.COM
I20260929 03:03:49.992413 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.976724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33386)
0929 03:03:49.976932 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.976953 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:49.977058 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:49.978041 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:49.978052 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:49.978570 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:49.978782 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:49.978796 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:49.980119 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:49.980128 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:49.981721 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:49.981737 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:49.982525 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:49.982537 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:49.982766 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:49.983382 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:49.983410 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:49.986204 (+ 2794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:49.989461 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:49.989468 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:49.989475 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:49.989962 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:49.990454 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:49.990458 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:49.990460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:49.990632 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:49.991505 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:49.991508 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:49.991614 (+ 106us) client_negotiation.cc:769] Sending connection context
0929 03:03:49.991723 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 03:03:49.991996 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 03:03:49.992743 31959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.976927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33386 (local address 127.8.208.125:35633)
0929 03:03:49.977631 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:49.977636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:49.977660 (+ 24us) server_negotiation.cc:407] Connection header received
0929 03:03:49.977702 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:49.977708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:49.977883 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:49.978002 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:49.978946 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:49.979978 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:49.981854 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:49.982369 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:49.982463 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:49.986422 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:49.986434 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:49.986447 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:49.986475 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:49.989354 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:49.990164 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:49.990176 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:49.990178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:49.990335 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:49.990767 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:49.990771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:49.990773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:49.991370 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:49.991562 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:49.992035 (+ 473us) server_negotiation.cc:299] Negotiation successful
0929 03:03:49.992241 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":418,"threads_started":1}
I20260929 03:03:49.995530 31954 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:03:49.996645 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.996188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49124)
0929 03:03:49.996361 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.996500 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260929 03:03:49.996899 31903 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.208.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.124.keytab.
Entry for principal kudu/127.8.208.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.124.keytab.
I20260929 03:03:49.998560 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:49.998143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49138)
0929 03:03:49.998302 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:49.998406 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 03:03:49.998788 9025 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1CUkeB/build/asan/bin/kudu
/tmp/dist-test-task1CUkeB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.124:38379
--webserver_interface=127.8.208.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.124.keytab
--principal=kudu/127.8.208.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.208.84:36555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.208.126:43567,127.8.208.125:35633,127.8.208.124:38379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:03:49.998798 31954 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:38379: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
I20260929 03:03:50.048120 31954 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } attempt: 1
I20260929 03:03:50.049402 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.048889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49150)
0929 03:03:50.049093 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.049252 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260929 03:03:50.050405 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.050065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49160)
0929 03:03:50.050227 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.050294 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260929 03:03:50.050614 31954 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:38379: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
I20260929 03:03:50.118831 31954 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } attempt: 2
I20260929 03:03:50.120102 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.119652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49164)
0929 03:03:50.119852 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.119943 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260929 03:03:50.121157 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.120816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49178)
0929 03:03:50.120973 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.121046 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 03:03:50.121405 31954 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:38379: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
I20260929 03:03:50.128968 31880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } attempt: 5
Sep 29 03:03:50 dist-test-slave-m1rv krb5kdc[31800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651029, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for kudu/127.8.208.125@KRBTEST.COM
I20260929 03:03:50.143759 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.129674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33390)
0929 03:03:50.129856 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.129878 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:50.129995 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:50.130400 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:50.130404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:50.130768 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:50.130965 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:50.130975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.132396 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.132400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:50.133366 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:50.133379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.134401 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.134406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:50.134566 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.135245 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:50.135262 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:50.137363 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:50.141378 (+ 4015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:50.141382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:50.141384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:50.141740 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:50.142204 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:50.142208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:50.142210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:50.142338 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:50.143044 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:50.143047 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:50.143173 (+ 126us) client_negotiation.cc:769] Sending connection context
0929 03:03:50.143287 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 03:03:50.143510 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260929 03:03:50.144057 31959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.129744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33390 (local address 127.8.208.125:35633)
0929 03:03:50.129914 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.129920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.129943 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:03:50.130120 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.130126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.130219 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.130337 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.131116 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.132226 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.133554 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.134221 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.134360 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.137571 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:50.137579 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:50.137586 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:50.137634 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:50.141235 (+ 3601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.141883 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.141891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.141896 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.142063 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.142452 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.142460 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.142464 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.142911 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:50.143091 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:50.143447 (+ 356us) server_negotiation.cc:299] Negotiation successful
0929 03:03:50.143710 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:03:50.145022 31880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:03:50.146093 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.145648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49188)
0929 03:03:50.145826 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.145929 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 03:03:50.147044 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.146681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49196)
0929 03:03:50.146825 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.146925 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 03:03:50.147245 31880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:38379: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
I20260929 03:03:50.198493 31880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } attempt: 1
I20260929 03:03:50.199733 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.199177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49200)
0929 03:03:50.199409 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.199522 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260929 03:03:50.200732 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.200378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49212)
0929 03:03:50.200523 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.200604 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 03:03:50.200939 31880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:38379: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
I20260929 03:03:50.212615 31954 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } attempt: 3
I20260929 03:03:50.213737 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.213301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49214)
0929 03:03:50.213502 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.213592 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 03:03:50.214773 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.214356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49228)
0929 03:03:50.214592 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.214665 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260929 03:03:50.214955 31954 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:38379: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
I20260929 03:03:50.269203 31880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } attempt: 2
I20260929 03:03:50.270512 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.269884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49234)
0929 03:03:50.270173 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.270290 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260929 03:03:50.271595 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.271226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49242)
0929 03:03:50.271386 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.271461 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260929 03:03:50.271799 31880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:38379: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
W20260929 03:03:50.330901 31961 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 03:03:50.331290 31961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:03:50.331420 31961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:03:50.331585 31961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:03:50.338644 31961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:03:50.338775 31961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:03:50.338876 31961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:03:50.338970 31961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:03:50.339059 31961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:03:50.348497 31961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.208.84:36555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.208.126:43567,127.8.208.125:35633,127.8.208.124:38379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/krb5kdc/kudu_127.8.208.124.keytab
--principal=kudu/127.8.208.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.208.124:38379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.208.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:55:41 UTC on ccycloud.quasar-gyuyoq.root.comops.site
build id 4679195
ASAN enabled
I20260929 03:03:50.349872 31961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:03:50.351137 31961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:03:50.358158 31954 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } attempt: 4
I20260929 03:03:50.359256 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.358785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49254)
0929 03:03:50.359014 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.359101 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 03:03:50.360239 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.359891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49270)
0929 03:03:50.360064 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.360137 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260929 03:03:50.360435 31954 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:38379: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
W20260929 03:03:50.362715 31966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:50.362955 31967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:03:50.363200 31969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:03:50.363546 31961 server_base.cc:1034] running on GCE node
I20260929 03:03:50.364254 31961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:03:50.365830 31961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:03:50.367084 31961 hybrid_clock.cc:648] HybridClock initialized: now 1790651030367029 us; error 52 us; skew 500 ppm
I20260929 03:03:50.367398 31961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 03:03:50 dist-test-slave-m1rv krb5kdc[31800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651030, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:03:50.370741 31961 init.cc:377] Logged in from keytab as kudu/127.8.208.124@KRBTEST.COM (short username kudu)
I20260929 03:03:50.372282 31961 webserver.cc:460] Webserver started at http://127.8.208.124:45927/ using document root <none> and password file <none>
I20260929 03:03:50.373003 31961 fs_manager.cc:359] Metadata directory not provided
I20260929 03:03:50.373102 31961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:03:50.373454 31961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:03:50.376361 31961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/data/instance:
uuid: "d355a160be2045028d9c211ff329c802"
format_stamp: "Formatted at 2026-09-29 03:03:50 on dist-test-slave-m1rv"
server_key: "ad5ba6f4b00ce12992d205ef1b3157eb"
server_key_iv: "4c877a5006a5e6d445c9ceaa3ba2fa42"
server_key_version: "encryptionkey@0"
I20260929 03:03:50.377151 31961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/wal/instance:
uuid: "d355a160be2045028d9c211ff329c802"
format_stamp: "Formatted at 2026-09-29 03:03:50 on dist-test-slave-m1rv"
server_key: "ad5ba6f4b00ce12992d205ef1b3157eb"
server_key_iv: "4c877a5006a5e6d445c9ceaa3ba2fa42"
server_key_version: "encryptionkey@0"
I20260929 03:03:50.378017 31880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } attempt: 3
I20260929 03:03:50.379079 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.378639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49286)
0929 03:03:50.378839 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.378922 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260929 03:03:50.380177 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.379855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49294)
0929 03:03:50.380002 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.380076 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260929 03:03:50.380461 31880 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.208.124:38379: Network error: Client connection negotiation failed: client connection to 127.8.208.124:38379: connect: Connection refused (error 111)
I20260929 03:03:50.383230 31961 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 03:03:50.386866 31976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:03:50.388275 31961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:03:50.388496 31961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/data,/tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/wal
uuid: "d355a160be2045028d9c211ff329c802"
format_stamp: "Formatted at 2026-09-29 03:03:50 on dist-test-slave-m1rv"
server_key: "ad5ba6f4b00ce12992d205ef1b3157eb"
server_key_iv: "4c877a5006a5e6d445c9ceaa3ba2fa42"
server_key_version: "encryptionkey@0"
I20260929 03:03:50.388733 31961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:03:50.408979 31961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:03:50.413324 31961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:03:50.413630 31961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:03:50.432912 31961 rpc_server.cc:307] RPC server started. Bound to: 127.8.208.124:38379
I20260929 03:03:50.432940 32027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.208.124:38379 every 8 connection(s)
I20260929 03:03:50.434311 31961 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/data/info.pb
I20260929 03:03:50.438198 32028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:03:50.443694 9025 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1CUkeB/build/asan/bin/kudu as pid 31961
I20260929 03:03:50.443892 9025 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1CUkeB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790650672582614-9025-0/minicluster-data/master-2/wal/instance
I20260929 03:03:50.444386 9025 external_mini_cluster.cc:1428] Setting key 87718cde9a26cb03b8f82fc5311b7dc1
I20260929 03:03:50.445238 32028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:03:50 dist-test-slave-m1rv krb5kdc[31800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651030, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.124@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
Sep 29 03:03:50 dist-test-slave-m1rv krb5kdc[31800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.124@KRBTEST.COM
I20260929 03:03:50.466586 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.448148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37060 (local address 127.8.208.126:43567)
0929 03:03:50.448347 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.448353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.449134 (+ 781us) server_negotiation.cc:407] Connection header received
0929 03:03:50.449581 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.449590 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.449673 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.449765 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.451509 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.452627 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.454952 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.455616 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.455745 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.460148 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:50.460156 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:50.460161 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:50.460210 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:50.463149 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.464031 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.464035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.464037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.464156 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.464647 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.464652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.464654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.465032 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:50.465222 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:50.466070 (+ 848us) server_negotiation.cc:299] Negotiation successful
0929 03:03:50.466241 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 03:03:50.467840 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.448173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.126:43567 (local address 127.0.0.1:37060)
0929 03:03:50.448974 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.449014 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:50.449356 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:50.449860 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:50.449866 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:50.450787 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:50.451333 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:50.451357 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.452808 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.452812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:50.454809 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:50.454825 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.455828 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.455832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:50.456143 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.457105 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:50.457143 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:50.459935 (+ 2792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:50.463278 (+ 3343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:50.463287 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:50.463302 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:50.463884 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:50.464268 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:50.464273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:50.464280 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:50.464531 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:50.465191 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:50.465201 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:50.465672 (+ 471us) client_negotiation.cc:769] Sending connection context
0929 03:03:50.465962 (+ 290us) client_negotiation.cc:240] Negotiation successful
0929 03:03:50.466488 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":79,"thread_start_us":382,"threads_started":1}
I20260929 03:03:50.467821 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.446365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49306 (local address 127.8.208.124:38379)
0929 03:03:50.447105 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.447117 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.447156 (+ 39us) server_negotiation.cc:407] Connection header received
0929 03:03:50.449142 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.449169 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.449477 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.449709 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.451521 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.453104 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.455105 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.455994 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.456139 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.461129 (+ 4990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:50.461144 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:50.461165 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:50.461237 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:50.464513 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.465361 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.465369 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.465372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.465521 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.465877 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.465882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.465885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.466311 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:50.466608 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:50.467068 (+ 460us) server_negotiation.cc:299] Negotiation successful
0929 03:03:50.467292 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":542,"thread_start_us":316,"threads_started":1}
I20260929 03:03:50.469584 32028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:03:50 dist-test-slave-m1rv krb5kdc[31800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651030, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.124@KRBTEST.COM for kudu/127.8.208.125@KRBTEST.COM
Sep 29 03:03:50 dist-test-slave-m1rv krb5kdc[31800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.126@KRBTEST.COM
I20260929 03:03:50.482915 31959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.470478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33404 (local address 127.8.208.125:35633)
0929 03:03:50.470663 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.470671 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.470859 (+ 188us) server_negotiation.cc:407] Connection header received
0929 03:03:50.471065 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.471070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.471156 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.471274 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.471954 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.472714 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.473934 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.474331 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.474458 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.477317 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:50.477321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:50.477324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:50.477353 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:50.480230 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.481099 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.481103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.481105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.481282 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.481706 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.481710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.481712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.481965 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:50.482082 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:50.482509 (+ 427us) server_negotiation.cc:299] Negotiation successful
0929 03:03:50.482658 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 03:03:50.483083 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.470417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.125:35633 (local address 127.0.0.1:33404)
0929 03:03:50.470761 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.470778 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:50.470910 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:50.471294 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:50.471298 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:50.471667 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:50.471814 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:50.471822 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.472845 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.472852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:50.473787 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:50.473798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.474477 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.474483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:50.474685 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.475219 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:50.475234 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:50.477089 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:50.480385 (+ 3296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:50.480388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:50.480390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:50.480984 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:50.481391 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:50.481396 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:50.481400 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:50.481593 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:50.482092 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:50.482099 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:50.482265 (+ 166us) client_negotiation.cc:769] Sending connection context
0929 03:03:50.482422 (+ 157us) client_negotiation.cc:240] Negotiation successful
0929 03:03:50.482692 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260929 03:03:50.484251 32028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:03:50.485919 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.471032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37062 (local address 127.8.208.126:43567)
0929 03:03:50.471197 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.471204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.471224 (+ 20us) server_negotiation.cc:407] Connection header received
0929 03:03:50.471411 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.471416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.471484 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.471583 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.472679 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.473479 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.474693 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.475281 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.475420 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.479798 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:50.479804 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:50.479808 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:50.479846 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:50.483099 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.483854 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.483881 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.483884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.484042 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.484411 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.484416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.484419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.484766 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:50.485061 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:50.485390 (+ 329us) server_negotiation.cc:299] Negotiation successful
0929 03:03:50.485639 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Sep 29 03:03:50 dist-test-slave-m1rv krb5kdc[31800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651030, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.124@KRBTEST.COM for kudu/127.8.208.124@KRBTEST.COM
Sep 29 03:03:50 dist-test-slave-m1rv krb5kdc[31800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.208.125@KRBTEST.COM
I20260929 03:03:50.499017 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.484938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49318)
0929 03:03:50.485111 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.485132 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:50.485266 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:50.485548 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:50.485551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:50.486080 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:50.486235 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:50.486245 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.487466 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.487472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:50.488855 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:50.488870 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.489650 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.489654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:50.489952 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.490481 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:50.490497 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:50.492896 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:50.496677 (+ 3781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:50.496680 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:50.496683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:50.497004 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:50.497403 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:50.497407 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:50.497411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:50.497534 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:50.498017 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:50.498021 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:50.498167 (+ 146us) client_negotiation.cc:769] Sending connection context
0929 03:03:50.498296 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 03:03:50.498585 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":48}
I20260929 03:03:50.499014 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.485128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.8.208.124:38379)
0929 03:03:50.485307 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.485315 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.485332 (+ 17us) server_negotiation.cc:407] Connection header received
0929 03:03:50.485372 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.485377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.485435 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.485535 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.486391 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.487298 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.488989 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.489526 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.489660 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.493084 (+ 3424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:50.493090 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:50.493098 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:50.493128 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:50.496512 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.497123 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.497131 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.497134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.497292 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.497631 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.497639 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.497642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.497898 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:50.498038 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:50.498391 (+ 353us) server_negotiation.cc:299] Negotiation successful
0929 03:03:50.498725 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":121,"server-negotiator.queue_time_us":82}
I20260929 03:03:50.503497 31959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.487481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33410 (local address 127.8.208.125:35633)
0929 03:03:50.487648 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.487651 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.487666 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:03:50.487800 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.487803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.487853 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.487924 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.489439 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.490473 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.492195 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.492752 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.492918 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.497999 (+ 5081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:50.498004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:50.498007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:50.498036 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:50.500907 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.501600 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.501607 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.501610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.501754 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.502146 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.502151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.502155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.502405 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:50.502681 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:50.502969 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 03:03:50.503205 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 03:03:50.504181 32028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802: Bootstrap starting.
I20260929 03:03:50.508221 32028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:50.508318 9025 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:03:50.509351 32028 log.cc:824] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:50.512107 32028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802: No bootstrap required, opened a new log
I20260929 03:03:50.510963 31855 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "311486e41c304d1cb8329201a2b9fa9b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37062
I20260929 03:03:50.519829 32028 raft_consensus.cc:348] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } }
I20260929 03:03:50.520251 32028 raft_consensus.cc:729] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d355a160be2045028d9c211ff329c802, State: Initialized, Role: FOLLOWER
I20260929 03:03:50.520699 32028 consensus_queue.cc:260] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } }
I20260929 03:03:50.521642 32034 sys_catalog.cc:455] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } } }
I20260929 03:03:50.522025 32034 sys_catalog.cc:458] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:50.522751 32028 sys_catalog.cc:564] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 03:03:50.532259 32045 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:03:50.532399 32045 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:03:50.529771 31961 master_runner.cc:186] Error getting master registration for 127.8.208.126:43567: OK, instance_id { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" instance_seqno: 1790651029417625 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 03:03:50.537346 31961 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 03:03:50.557659 31880 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } attempt: 4
Sep 29 03:03:50 dist-test-slave-m1rv krb5kdc[31800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651029, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.126@KRBTEST.COM for kudu/127.8.208.124@KRBTEST.COM
I20260929 03:03:50.573741 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.558400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49328)
0929 03:03:50.558665 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.558681 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:50.558826 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:50.559244 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:50.559251 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:50.559813 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:50.560028 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:50.560043 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.561528 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.561533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:50.563016 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:50.563032 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.564152 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.564158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:50.564387 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.565059 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:50.565077 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:50.567796 (+ 2719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:50.571067 (+ 3271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:50.571074 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:50.571078 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:50.571449 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:50.571866 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:50.571871 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:50.571874 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:50.572035 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:50.572699 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:50.572704 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:50.572886 (+ 182us) client_negotiation.cc:769] Sending connection context
0929 03:03:50.573035 (+ 149us) client_negotiation.cc:240] Negotiation successful
0929 03:03:50.573337 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260929 03:03:50.573745 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.558452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49328 (local address 127.8.208.124:38379)
0929 03:03:50.558658 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.558664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.558735 (+ 71us) server_negotiation.cc:407] Connection header received
0929 03:03:50.559001 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.559007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.559089 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.559202 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.560199 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.561360 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.563170 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.564000 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.564129 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.567999 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:50.568005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:50.568010 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:50.568054 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:50.570928 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.571570 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.571576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.571579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.571754 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.572152 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.572156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.572159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.572578 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:50.572740 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:50.573127 (+ 387us) server_negotiation.cc:299] Negotiation successful
0929 03:03:50.573360 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 03:03:50.578828 31880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b: Bootstrap starting.
I20260929 03:03:50.582074 31880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:50.583153 31880 log.cc:824] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:50.585927 31880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b: No bootstrap required, opened a new log
I20260929 03:03:50.591238 31880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } }
I20260929 03:03:50.591589 31880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 311486e41c304d1cb8329201a2b9fa9b, State: Initialized, Role: FOLLOWER
I20260929 03:03:50.592075 31880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } }
I20260929 03:03:50.593108 32048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } } }
I20260929 03:03:50.593657 32048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:50.594306 31880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:50.594444 31855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 03:03:50.594656 31855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:50.598116 31855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } }
I20260929 03:03:50.599659 31855 leader_election.cc:288] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [CANDIDATE]: Term 1 election: Requested vote from peers 6049a9d348934151ba369400ebd5428f (127.8.208.125:35633), d355a160be2045028d9c211ff329c802 (127.8.208.124:38379)
I20260929 03:03:50.601652 32003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "311486e41c304d1cb8329201a2b9fa9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d355a160be2045028d9c211ff329c802"
I20260929 03:03:50.602240 32003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:50.606808 32003 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 311486e41c304d1cb8329201a2b9fa9b in term 1.
I20260929 03:03:50.607787 31829 leader_election.cc:302] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 311486e41c304d1cb8329201a2b9fa9b, d355a160be2045028d9c211ff329c802; no voters:
I20260929 03:03:50.608285 32048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:03:50.609540 32048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [term 1 LEADER]: Becoming Leader. State: Replica: 311486e41c304d1cb8329201a2b9fa9b, State: Running, Role: LEADER
I20260929 03:03:50.610180 32048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } }
I20260929 03:03:50.614782 32057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 311486e41c304d1cb8329201a2b9fa9b. Latest consensus state: current_term: 1 leader_uuid: "311486e41c304d1cb8329201a2b9fa9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } } }
I20260929 03:03:50.615265 32057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:50.616591 32060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:03:50.635509 32060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:03:50.654497 32071 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 03:03:50.655898 32003 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [term 1 FOLLOWER]: Refusing update from remote peer 311486e41c304d1cb8329201a2b9fa9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260929 03:03:50.657387 31834 tablet.cc:2369] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:03:50.659070 32057 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:03:50.659634 31954 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } attempt: 5
W20260929 03:03:50.667788 31982 tablet.cc:2369] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:03:50.675920 31884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.627247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37064 (local address 127.8.208.126:43567)
0929 03:03:50.627446 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.627453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.643677 (+ 16224us) server_negotiation.cc:407] Connection header received
0929 03:03:50.646083 (+ 2406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.646089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.646175 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.646249 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.652546 (+ 6297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.653744 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.655848 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.656596 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.656698 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.665907 (+ 9209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:50.665915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:50.665921 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:50.665964 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:50.670021 (+ 4057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.672180 (+ 2159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.672186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.672190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.672347 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.673611 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.673620 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.673625 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.674074 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:50.674371 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:50.675074 (+ 703us) server_negotiation.cc:299] Negotiation successful
0929 03:03:50.675400 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 03:03:50.679756 31959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.629084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33422 (local address 127.8.208.125:35633)
0929 03:03:50.629271 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.629277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.649798 (+ 20521us) server_negotiation.cc:407] Connection header received
0929 03:03:50.650058 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.650064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.650152 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.650219 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.655840 (+ 5621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.656929 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.661310 (+ 4381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.662076 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.662193 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.667832 (+ 5639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:50.667838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:50.667843 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:50.667889 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:50.672722 (+ 4833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.674409 (+ 1687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.674442 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.674445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.674716 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.675475 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.675480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.675509 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.676437 (+ 928us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:50.677019 (+ 582us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:50.678679 (+ 1660us) server_negotiation.cc:299] Negotiation successful
0929 03:03:50.679087 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 03:03:50.662556 31813 master_runner.cc:186] Error getting master registration for 127.8.208.125:35633: OK, instance_id { permanent_uuid: "6049a9d348934151ba369400ebd5428f" instance_seqno: 1790651029921256 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 03:03:50.686771 31813 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 03:03:50.694006 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.644203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49332 (local address 127.8.208.124:38379)
0929 03:03:50.644374 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.644380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.649961 (+ 5581us) server_negotiation.cc:407] Connection header received
0929 03:03:50.650190 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.650198 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.650288 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.650357 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.657524 (+ 7167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.658579 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.663659 (+ 5080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.664445 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.664548 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.678525 (+ 13977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:50.678532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:50.678537 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:50.678581 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:50.682394 (+ 3813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.684536 (+ 2142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.684542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.684546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.684738 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.685196 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.685202 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.685208 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.685654 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:50.693043 (+ 7389us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:50.693268 (+ 225us) server_negotiation.cc:299] Negotiation successful
0929 03:03:50.693533 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Sep 29 03:03:50 dist-test-slave-m1rv krb5kdc[31800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651029, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.208.125@KRBTEST.COM for kudu/127.8.208.124@KRBTEST.COM
I20260929 03:03:50.705266 32034 sys_catalog.cc:455] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 311486e41c304d1cb8329201a2b9fa9b. Latest consensus state: current_term: 1 leader_uuid: "311486e41c304d1cb8329201a2b9fa9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } } }
I20260929 03:03:50.705633 32034 sys_catalog.cc:458] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:50.709730 32069 mvcc.cc:205] Tried to move back new op lower bound from 7334506621550149632 to 7334506621385060352. Current Snapshot: MvccSnapshot[applied={T|T < 7334506621550149632}]
I20260929 03:03:50.712817 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.660144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.208.124:38379 (local address 127.0.0.1:49348)
0929 03:03:50.660488 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0929 03:03:50.660508 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:03:50.660610 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:03:50.695389 (+ 34779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:03:50.695398 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:03:50.696080 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:03:50.696353 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:50.696369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.698615 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.698622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:50.699932 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:03:50.699946 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.700933 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.700939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:03:50.701198 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.701887 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:03:50.701910 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:03:50.704734 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:03:50.710018 (+ 5284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:50.710024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:50.710028 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:50.710535 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:50.710950 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:03:50.710955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:03:50.710959 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:03:50.711133 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:03:50.711821 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:03:50.711826 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:03:50.711967 (+ 141us) client_negotiation.cc:769] Sending connection context
0929 03:03:50.712138 (+ 171us) client_negotiation.cc:240] Negotiation successful
0929 03:03:50.712426 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260929 03:03:50.714166 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:50.660950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49348 (local address 127.8.208.124:38379)
0929 03:03:50.694314 (+ 33364us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.694323 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.694346 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:03:50.694411 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.694419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.694513 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.694639 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.697098 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.698424 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.700101 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:50.700772 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:50.700900 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:50.705031 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:03:50.705040 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:03:50.705048 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:03:50.705093 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:03:50.709760 (+ 4667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.710676 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.710682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.710685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.710821 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:03:50.711260 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:03:50.711265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:03:50.711269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:03:50.711609 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:03:50.711751 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:50.713532 (+ 1781us) server_negotiation.cc:299] Negotiation successful
0929 03:03:50.713744 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33216,"thread_start_us":36636,"threads_started":1}
I20260929 03:03:50.720248 31954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f: Bootstrap starting.
I20260929 03:03:50.725335 32073 mvcc.cc:205] Tried to move back new op lower bound from 7334506621550149632 to 7334506621385060352. Current Snapshot: MvccSnapshot[applied={T|T < 7334506621550149632}]
I20260929 03:03:50.727110 31954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f: Neither blocks nor log segments found. Creating new log.
I20260929 03:03:50.727919 32048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "311486e41c304d1cb8329201a2b9fa9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } } }
I20260929 03:03:50.728260 32048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:50.729681 31954 log.cc:824] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f: Log is configured to *not* fsync() on all Append() calls
I20260929 03:03:50.735965 32060 catalog_manager.cc:1347] Generated new cluster ID: 3f96623a712845ffa886970e2a7704b1
I20260929 03:03:50.736074 32060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:03:50.750836 32034 sys_catalog.cc:455] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "311486e41c304d1cb8329201a2b9fa9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } } }
I20260929 03:03:50.751174 32034 sys_catalog.cc:458] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:50.751091 31954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f: No bootstrap required, opened a new log
I20260929 03:03:50.766815 31954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } }
I20260929 03:03:50.767344 31954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6049a9d348934151ba369400ebd5428f, State: Initialized, Role: FOLLOWER
I20260929 03:03:50.767689 32060 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:03:50.767975 31954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } }
I20260929 03:03:50.775266 32060 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:03:50.777103 31954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:03:50.777132 31929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "311486e41c304d1cb8329201a2b9fa9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6049a9d348934151ba369400ebd5428f"
I20260929 03:03:50.777634 31929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:03:50.775126 32078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } } }
I20260929 03:03:50.778863 32078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:50.781162 31929 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 311486e41c304d1cb8329201a2b9fa9b in term 1.
I20260929 03:03:50.782266 31928 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [term 1 FOLLOWER]: Refusing update from remote peer 311486e41c304d1cb8329201a2b9fa9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:03:50.786571 32057 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:03:50.816320 32060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b: Generated new TSK 0
I20260929 03:03:50.817670 32060 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:03:50.822249 32082 mvcc.cc:205] Tried to move back new op lower bound from 7334506621972529152 to 7334506621385060352. Current Snapshot: MvccSnapshot[applied={T|T < 7334506621550149632}]
I20260929 03:03:50.829613 32048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "311486e41c304d1cb8329201a2b9fa9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } } }
I20260929 03:03:50.829948 32048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 311486e41c304d1cb8329201a2b9fa9b [sys.catalog]: This master's current role is: LEADER
I20260929 03:03:50.830178 32078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 311486e41c304d1cb8329201a2b9fa9b. Latest consensus state: current_term: 1 leader_uuid: "311486e41c304d1cb8329201a2b9fa9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } } }
I20260929 03:03:50.830510 32078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:50.845592 32088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "311486e41c304d1cb8329201a2b9fa9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "311486e41c304d1cb8329201a2b9fa9b" member_type: VOTER last_known_addr { host: "127.8.208.126" port: 43567 } } peers { permanent_uuid: "6049a9d348934151ba369400ebd5428f" member_type: VOTER last_known_addr { host: "127.8.208.125" port: 35633 } } peers { permanent_uuid: "d355a160be2045028d9c211ff329c802" member_type: VOTER last_known_addr { host: "127.8.208.124" port: 38379 } } }
I20260929 03:03:50.846045 32088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:03:50.869705 32096 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260929 03:03:50.905586 31884 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:50.902717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.8.208.126:43567)
0929 03:03:50.902963 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.902969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.903761 (+ 792us) server_negotiation.cc:407] Connection header received
0929 03:03:50.904050 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.904056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.904141 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.904215 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.905334 (+ 1119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37078: BlockingRecv error: recv got EOF from 127.0.0.1:37078 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260929 03:03:50.907363 31959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:50.903500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.8.208.125:35633)
0929 03:03:50.903696 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.903702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.906097 (+ 2395us) server_negotiation.cc:407] Connection header received
0929 03:03:50.906233 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.906239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.906330 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.906450 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.907158 (+ 708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33438: BlockingRecv error: recv got EOF from 127.0.0.1:33438 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260929 03:03:50.909474 32031 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:50.904481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49358 (local address 127.8.208.124:38379)
0929 03:03:50.904655 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.904661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.907920 (+ 3259us) server_negotiation.cc:407] Connection header received
0929 03:03:50.908163 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.908168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.908241 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.908306 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.909264 (+ 958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49358: BlockingRecv error: recv got EOF from 127.0.0.1:49358 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260929 03:03:50.924007 31959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:50.921860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33444 (local address 127.8.208.125:35633)
0929 03:03:50.922030 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.922036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.922476 (+ 440us) server_negotiation.cc:407] Connection header received
0929 03:03:50.922719 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.922725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.922848 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.922955 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.923747 (+ 792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33444: BlockingRecv error: recv got EOF from 127.0.0.1:33444 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260929 03:03:50.968336 31959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:50.962762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.8.208.125:35633)
0929 03:03:50.962946 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.962952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.966711 (+ 3759us) server_negotiation.cc:407] Connection header received
0929 03:03:50.966931 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.966939 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.967037 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.967170 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.968119 (+ 949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33458: BlockingRecv error: recv got EOF from 127.0.0.1:33458 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260929 03:03:50.979274 31959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:50.974495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33468 (local address 127.8.208.125:35633)
0929 03:03:50.974670 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.974676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.977719 (+ 3043us) server_negotiation.cc:407] Connection header received
0929 03:03:50.977912 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.977919 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.978038 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.978157 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.979026 (+ 869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33468: BlockingRecv error: recv got EOF from 127.0.0.1:33468 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260929 03:03:50.988852 31959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:50.986397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33480 (local address 127.8.208.125:35633)
0929 03:03:50.986607 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:50.986614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:50.987105 (+ 491us) server_negotiation.cc:407] Connection header received
0929 03:03:50.987509 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:50.987515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:50.987616 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:50.987725 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:50.988580 (+ 855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33480: BlockingRecv error: recv got EOF from 127.0.0.1:33480 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260929 03:03:51.002636 31959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:51.000649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33484 (local address 127.8.208.125:35633)
0929 03:03:51.000845 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:51.000851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:51.001011 (+ 160us) server_negotiation.cc:407] Connection header received
0929 03:03:51.001314 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:51.001320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:51.001426 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:51.001549 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:51.002402 (+ 853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33484: BlockingRecv error: recv got EOF from 127.0.0.1:33484 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260929 03:03:51.025455 31959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:51.022806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33490 (local address 127.8.208.125:35633)
0929 03:03:51.023050 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:51.023057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:51.023680 (+ 623us) server_negotiation.cc:407] Connection header received
0929 03:03:51.023892 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:51.023901 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:51.024034 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:51.024174 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:51.025136 (+ 962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33490: BlockingRecv error: recv got EOF from 127.0.0.1:33490 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260929 03:03:51.065683 31959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:51.063269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33496 (local address 127.8.208.125:35633)
0929 03:03:51.063451 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:51.063458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:51.063995 (+ 537us) server_negotiation.cc:407] Connection header received
0929 03:03:51.064213 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:51.064219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:51.064325 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:51.064454 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:51.065361 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33496: BlockingRecv error: recv got EOF from 127.0.0.1:33496 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260929 03:03:51.135046 31959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:51.133107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33504 (local address 127.8.208.125:35633)
0929 03:03:51.133327 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:51.133334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:51.133716 (+ 382us) server_negotiation.cc:407] Connection header received
0929 03:03:51.133880 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:51.133885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:51.133975 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:51.134093 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:51.134825 (+ 732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33504: BlockingRecv error: recv got EOF from 127.0.0.1:33504 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260929 03:03:51.268309 31959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:51.266266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.8.208.125:35633)
0929 03:03:51.266447 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:51.266455 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:51.266968 (+ 513us) server_negotiation.cc:407] Connection header received
0929 03:03:51.267128 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:51.267133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:51.267208 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:51.267296 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:51.268042 (+ 746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33510: BlockingRecv error: recv got EOF from 127.0.0.1:33510 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260929 03:03:51.529451 31959 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:03:51.527649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33518 (local address 127.8.208.125:35633)
0929 03:03:51.527881 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:51.527887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:51.528106 (+ 219us) server_negotiation.cc:407] Connection header received
0929 03:03:51.528273 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:51.528277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:51.528366 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:51.528475 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:03:51.529242 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33518: BlockingRecv error: recv got EOF from 127.0.0.1:33518 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 03:03:51.534265 32045 catalog_manager.cc:1259] Loaded cluster ID: 3f96623a712845ffa886970e2a7704b1
I20260929 03:03:51.534387 32045 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802: loading cluster ID for follower catalog manager: success
I20260929 03:03:51.536799 32045 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802: acquiring CA information for follower catalog manager: success
I20260929 03:03:51.538604 32045 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d355a160be2045028d9c211ff329c802: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 03:03:51.875485 32096 catalog_manager.cc:1259] Loaded cluster ID: 3f96623a712845ffa886970e2a7704b1
I20260929 03:03:51.875630 32096 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f: loading cluster ID for follower catalog manager: success
I20260929 03:03:51.879554 32096 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f: acquiring CA information for follower catalog manager: success
I20260929 03:03:51.882153 32096 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6049a9d348934151ba369400ebd5428f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 03:03:52.058282 32161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:52.051296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33532 (local address 127.8.208.125:35633)
0929 03:03:52.052229 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:52.052238 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:52.052255 (+ 17us) server_negotiation.cc:407] Connection header received
0929 03:03:52.052322 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:52.052326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:52.052475 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:52.052627 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:52.053133 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:52.054130 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:52.055616 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:52.056162 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:52.056325 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:52.056901 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:52.057579 (+ 678us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:52.057772 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:52.057856 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 03:03:52.057957 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":571,"threads_started":1}
I20260929 03:03:52.077214 32167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:52.070581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49362 (local address 127.8.208.124:38379)
0929 03:03:52.071332 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:52.071341 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:52.071364 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:03:52.071489 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:52.071519 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:52.071714 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:52.071978 (+ 264us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:52.072380 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:52.073252 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:52.074716 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:52.075180 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:52.075336 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:52.075917 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:52.076429 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:52.076620 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:52.076733 (+ 113us) server_negotiation.cc:299] Negotiation successful
0929 03:03:52.076823 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":485,"threads_started":1}
I20260929 03:03:52.098481 32161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:52.083930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33534 (local address 127.8.208.125:35633)
0929 03:03:52.084132 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:52.084138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:52.090074 (+ 5936us) server_negotiation.cc:407] Connection header received
0929 03:03:52.090384 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:52.090394 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:52.090528 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:52.090630 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:52.091528 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:52.092783 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:52.095827 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:52.096554 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:52.096734 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:52.097491 (+ 757us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:52.097638 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:52.097712 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:52.098003 (+ 291us) server_negotiation.cc:299] Negotiation successful
0929 03:03:52.098106 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 03:03:52.098819 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:52.084117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37094 (local address 127.8.208.126:43567)
0929 03:03:52.089596 (+ 5479us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:52.089604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:52.089629 (+ 25us) server_negotiation.cc:407] Connection header received
0929 03:03:52.089758 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:52.089764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:52.089992 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:52.090891 (+ 899us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:52.091091 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:52.092432 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:52.095359 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:52.096130 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:52.096969 (+ 839us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:52.097193 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:52.097710 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:52.098283 (+ 573us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:52.098343 (+ 60us) server_negotiation.cc:299] Negotiation successful
0929 03:03:52.098469 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5325,"thread_start_us":5208,"threads_started":1}
I20260929 03:03:52.103268 32167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:03:52.090335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49372 (local address 127.8.208.124:38379)
0929 03:03:52.090515 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 03:03:52.090521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:03:52.095454 (+ 4933us) server_negotiation.cc:407] Connection header received
0929 03:03:52.095739 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:03:52.095746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:03:52.095871 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:03:52.095954 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:03:52.096983 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:52.098146 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:52.101142 (+ 2996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:03:52.101674 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:03:52.101836 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:03:52.102363 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:03:52.102554 (+ 191us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:03:52.102658 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 03:03:52.102854 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 03:03:52.102943 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 03:03:52.105154 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 31813
I20260929 03:03:52.106340 31875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:52.224602 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 31813
W20260929 03:03:52.253527 31977 connection.cc:535] server connection from 127.0.0.1:49328 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 03:03:52.254276 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 31887
I20260929 03:03:52.255578 31949 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:52.370056 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 31887
I20260929 03:03:52.386161 9025 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1CUkeB/build/asan/bin/kudu pid 31961
I20260929 03:03:52.387261 32023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 03:03:52.493269 9025 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1CUkeB/build/asan/bin/kudu with pid 31961
2026-09-29T03:03:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4862 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9345 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (359879 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST